IEEE Signal Processing Magazine - January 2018 - 135

compression. He also has published many works regarding the
applications of deep learning in computer vision and natural
language processing.
Duo Wang (d-wang15@mails.tsinghua.edu.cn) received the
B.S. degree in automation from the Harbin Institute of
Technology, China, in 2015, where he is currently pursuing his
Ph.D. degree in the Department of Automation, Tsinghua
University. His research interests are deep/machine learning and
their applications in computer vision and robotics vision.
Pan Zhou (panzhou@hust.edu.cn) received his B.S. degree
in the Advanced Class of Huazhong University of Science and
Technology (HUST), Wuhan China, and his M.S. degree in electronics and information engineering from the same university in
2006 and 2008, respectively. He received his Ph.D. degree from
the School of Electrical and Computer Engineering at the
Georgia Institute of Technology, Atlanta in 2011. Currently, he is
an associate professor with School of Electronic Information and
Communications, HUST. His research interests include big data
analytics and machine learning, security and privacy, and information networks.
Tao Zhang (taozhang@mail.tsinghua.edu.cn) received his
B.S., M.S., and Ph.D. degrees from Tsinghua University,
Beijing, China, in 1993, 1995, and 1999, respectively, and his
Ph.D. degree from Saga University, Japan, in 2002, all in control engineering. He is a professor with the Department of
Automation, Tsinghua University. His current research interests include artificial intelligence, robotics, image processing,
control theory, and control of spacecraft.

References

[1] A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep
convolutional neural networks," in Proc. Conf. Neural Information Processing
Systems, 2012, pp. 1097-1105.
[2] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, "Deepface: Closing the gap to
human-level performance in face verification," in Proc. IEEE Conf. Computer
Vision Pattern Recognition, 2014, pp. 1701-1708.
[3] Y. Sun, X. Wang, and X. Tang, "Deeply learned face representations are sparse,
selective, and robust," in Proc. IEEE Conf. Computer Vision Pattern Recognition,
2015, pp. pp. 2892-2900.
[4] J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M.
Ranzato, A. Senior, P. Tucker, K. Yang, and A. Ng, "Large scale distributed deep
networks," in Proc. Conf. Neural Information Processing Systems, 2012, pp.
1223-1231.
[5] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Computing Res. Repository, vol. abs/1512.03385, 2015. [Online]. Available:
https://arxiv.org/pdf/1512.03385.pdf
[6] Y. Gong, L. Liu, M. Yang, and L. D. Bourdev, "Compressing deep convolutional
networks using vector quantization," Computing Res. Repository, vol.
abs/1412.6115, 2014. [Online]. Available: https://arxiv.org/pdf/1412.6115.pdf
[7] Y. W. Q. H. Jiaxiang Wu, C. Leng, and J. Cheng, "Quantized convolutional neural networks for mobile devices," in Proc. IEEE Conf. Computer Vision Pattern
Recognition, 2016, pp. 4820-4828.

[12] C. Zhu, S. Han, H. Mao, and W. J. Dally, "Trained ternary quantization," arXiv
Preprint, arXiv:1612.01064, 2016.
[13] M. Courbariaux, Y. Bengio, and J. David, "Binaryconnect: Training deep neural networks with binary weights during propagations," in Proc. Advances Neural
Information Processing Systems Annu. Conf., 2015, pp. 3123-3131.
[14] M. Courbariaux and Y. Bengio, "Binarynet: Training deep neural networks
with weights and activations constrained to +1 or −1," Computing Res. Repository,
vol. abs/1602.02830, 2016. [Online]. Available: https://arxiv.org/abs/1602.02830
[15] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, "Xnor-net: Imagenet
classification using binary convolutional neural networks," in Proc. European Conf.
Computer Vision, 2016, pp. 525-542.
[16] P. Merolla, R. Appuswamy, J. V. Arthur, S. K. Esser, and D. S. Modha, "Deep
neural networks are robust to weight binarization and other non-linear distortions,"
Computing Res. Repository, vol. abs/1606.01981, 2016. [Online]. Available: https://
arxiv.org/abs/1606.01981
[17] L. Hou, Q. Yao, and J. T. Kwok, "Loss-aware binarization of deep networks,"
Computing Res. Repository, vol. abs/1611.01600, 2016. [Online]. Available: https://
arxiv.org/abs/1611.01600
[18] Z. Lin, M. Courbariaux, R. Memisevic, and Y. Bengio, "Neural networks with
few multiplications," Computing Res. Repository, vol. abs/1510.03009, 2015.
[Online]. Available: https://arxiv.org/abs/1510.03009
[19] S. J. Hanson and L. Y. Pratt, "Comparing biases for minimal network construction with back-propagation," Adv. Neural Inform. Process. Syst. 1, 1989, pp.
177-185.
[20] Y. L. Cun, J. S. Denker, and S. A. Solla, "Advances in neural information processing systems 2," in Optimal Brain Damage, D. S. Touretzky, Ed. San Mateo,
CA: Morgan Kaufmann, 1990, pp. 598-605.
[21] B. Hassibi, D. G. Stork, and S. C. R. Com, "Second order derivatives for
network pruning: Optimal brain surgeon," in Advances in Neural Information
Processing Systems, vol. 5. San Mateo, CA: Morgan Kaufmann, 1993, pp. 164-
171.
[22] S. Srinivas and R. V. Babu, "Data-free parameter pruning for deep neural networks," in Proc. British Machine Vision Conf., 2015, pp. 31.1-31.12.
[23] S. Han, J. Pool, J. Tran, and W. J. Dally, "Learning both weights and connections for efficient neural networks," in Proc. 28th Int. Conf. Neural Information
Processing Systems, 2015, pp. 1135-1143.
[24] W. Chen, J. Wilson, S. Tyree, K. Q. Weinberger, and Y. Chen, "Compressing
neural networks with the hashing trick," in Proc. Machine Learning Research
Workshop Conf., 2015, pp. 2285-2294.
[25] K. Ullrich, E. Meeds, and M. Welling, "Soft weight-sharing for neural network
compression," Computing Res. Repository, vol. abs/1702.04008, 2017. [Online].
Available: https://arxiv.org/abs/1702.04008
[26] V. Lebedev and V. S. Lempitsky, "Fast convnets using group-wise brain damage," in Proc. IEEE Conf. Computer Vision Pattern Recognition, 2016, pp. 2554-
2564.
[27] H. Zhou, J. M. Alvarez, and F. Porikli, "Less is more: Towards compact
CNNs," in Proc. European Conf. Computer Vision, 2016, pp. 662-677.
[28] W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, "Learning structured sparsity in
deep neural networks," Adv. Neural Inform. Process. Syst., vol. 29, pp. 2074-2082,
2016.
[29] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, "Pruning filters for
efficient convnets," Computing Res. Repository, vol. abs/1608.08710, 2016.
[Online]. Available: https://arxiv.org/abs/1608.08710
[30] Y. Cheng, F. X. Yu, R. Feris, S. Kumar, A. Choudhary, and S.-F. Chang, "An
exploration of parameter redundancy in deep networks with circulant projections," in
Proc. Int. Conf. Computer Vision, 2015, pp. 2857-2865.
[31] Z. Yang, M. Moczulski, M. Denil, N. de Freitas, A. Smola, L. Song, and Z.
Wang, "Deep fried convnets," in Proc. Int. Conf. Computer Vision, 2015, pp. 1476-
1483.

[8] V. Vanhoucke, A. Senior, and M. Z. Mao, "Improving the speed of neural networks on cpus," in Proc. Conf. Neural Information Processing Systems Deep
Learning and Unsupervised Feature Learning Workshop, 2011.

[32] V. Sindhwani, T. Sainath, and S. Kumar. (2015). Structured transforms for
small-footprint deep learning. Advances in Neural Information Processing
Systems, 28, pp. 3088-3096. [Online]. Available: http://papers.nips.cc/paper/5869structured-transforms-for-small-footprint-deep-learning.pdf

[9] S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, "Deep learning
with limited numerical precision," in Proc. 32nd Int. Conf. Machine Learning,
2015, vol. 37, pp. 1737-1746.

[33] J. Chun and T. Kailath, Generalized Displacement Structure for BlockToeplitz, Toeplitz-Block, and Toeplitz-Derived Matrices. Berlin, Germany:
Springer, 1991, pp. 215-236.

[10] S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural
networks with pruning, trained quantization and Huffman coding," in Proc. Int.
Conf. Learning Representations, 2016.

[34] M. V. Rakhuba and I. V. Oseledets. (2015). Fast multidimensional convolution
in low-rank tensor formats via cross approximation. SIAM J. Sci. Comput., 37(2).
[Online]. Available: http://dx.doi.org/10.1137/140958529

[11] Y. Choi, M. El-Khamy, and J. Lee, "Towards the limit of network quantization," Computing Res. Repository, vol. abs/1612.01543, 2016. [Online]. Available:
https://arxiv.org/abs/1612.01543

[35] R. Rigamonti, A. Sironi, V. Lepetit, and P. Fua, "Learning separable filters,"
in Proc. IEEE Conf. Computer Vision Pattern Recognition, 2013, pp. 2754-
2761.

IEEE SIGNAL PROCESSING MAGAZINE

|

January 2018

|

135
Table of Contents for the Digital Edition of IEEE Signal Processing Magazine - January 2018
