IEEE Solid-State Circuits Magazine - Fall 2017 - 65

IEEE Conf. Computer Vision and Pattern
Recognition, 2015, pp. 1-9.
[6] F. Chollet, "Xception: Deep learning with
depthwise separable convolutions," arXiv
Preprint, arXiv:1610.02357, 2016.
[7] F. Iandola, M. Moskewicz, S. Karayev,
R. Girshick, T. Darrell, and K. Keutzer,
"Densenet: Implementing efficient convnet descriptor pyramids," arXiv Preprint,
arXiv:1404.1869, 2014.
[8] F. A. Gers, J. Schmidhuber, and F. Cummins, "Learning to forget: Continual prediction with LSTM," Neural Comput., vol.
12, no. 10, pp. 2451-2471, 2000.
[9] N. P. Jouppi, et al. "In-datacenter performance analysis of a tensor processing
unit," arXiv Preprint, arXiv:1704.04760,
2017.
[10] D. Shin, J. Lee, J. Lee, and H.-J. Yoo, "DNPU:
An 8.1 TOPS/W reconfigurable CNN-RNN
processor for general-purpose deep neural networks," in Proc. IEEE Int. Solid-State
Circuits Conf., 2017, pp. 240-241.
[11] Y.-H. Chen, J. Emer, and V. Sze, "Eyeriss:
A spatial architecture for energy-efficient dataflow for convolutional neural
networks," in Proc. IEEE Annu. Int. Symp.
Computer Architecture, 2016, pp. 367-
379.
[12] M. Peemen, et al. "Memory-centric accelerator design for convolutional neural
networks," in Proc. IEEE 31st Int. Conf.
Computer Design, 2013, pp. 13-19.
[13] L. Cecconi, S. Smets, L. Benini, and M.
Verhelst, "Optimal tiling strategy for
memory bandwidth reduction for Cnns:
Advanced concepts for intelligent vision
systems," Ph.D. dissertation, Univ. Bologna 2017.
[14] Y.-H. Chen, T. Krishna, J. Emer, and V. Sze.
"Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional
neural networks," in Proc. IEEE Int. SolidState Circuits Conf., 2016, pp. 262-263.
[15] H. T. Kung, "Systolic algorithms for the
CMU WARP processor," Research Showcase @ CMU, 1984.
[16] A. Shafiee, A. Nag, N. Muralimanohar, R.
Balasubramonian, J. P. Strachan, M. Hu,
R. S. Williams, and V. Srikumar, "ISAAC: A
convolutional neural network accelerator
with in-situ analog arithmetic in crossbars," in Proc. 43rd Int. Symp. Computer
Architecture, 2016, pp.14-26.
[17] P Gysel, M. Motamedi, and S. Ghiasi,
"Hardware-oriented approximation of
convolutional neural networks," in Proc.
Workshop Contribution to Int. Conf. Learning Representations, 2016.
[18] B. Moons, B. De Brabandere, L. Van Gool,
and M. Verhelst, "Energy-efficient ConvNets through approximate computing,"
in Proc. IEEE Winter Conf. Applications
Computer Vision, 2016, pp. 1-8.
[19] I. Hubara, M. Courbariaux, D. Soudry, R.
El-Yaniv, and Y. Bengio, "Quantized neural networks: Training neural networks
with low precision weights and activations," arXiv preprint, arXiv:1609.07061,
2016.
[20] M. Rastegari, V. Ordonez, J. Redmon, and
A. Farhadi, "XNOR-Net: Imagenet classification using binary convolutional neural
networks," in Proc. European Conf. Computer Vision, 2016, pp. 525-542.
[21] I. Hubara, M. Courbariaux, D. Soudry, R.
El-Yaniv, and Y. Bengio, "Binarized Neural
networks in advances" in Neural Information Processing Systems 29, D. D. Lee, M.
Sugiyama, U. V. Luxburg, I. Guyon, and R.
Garnett, Eds. Curran Assoc., Inc. 2016, pp.
4107-4115.
[22] R. Andri, L. Cavigelli, D. Rossi, and L. Benini, "YodaNN: An ultra-low power convo-

lutional neural network accelerator based
on binary weights," in Proc. IEEE VLSI
Computer Society Annu. Symp., July 2016,
pp. 236-241.
[23] B. Moons and M. Verhelst, "A 0.3-2.6 TOPS/W
precision-scalable processor for real-time
large-scale ConvNets," in Proc. IEEE Symp.
VLSI Circuits, 2016, pp. 1-2.
[24] B. Moons, et al. "Envision: A 0.26-to-10
TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable convolutional neural network processor in 28
nm FDSOI," in Proc. IEEE Int. Solid-State
Circuits Conf., 2017, pp. 246-257.
[25] B. Moons, R. Uytterhoeven, W. Dehaene,
and M. Verhelst, "DVAFS: Trading computational accuracy for energy through
dynamic-voltage-accuracy-frequencyscaling," in Proc. Conf. Design, Automation and Test in Europe, Lausanne, 2017,
pp. 488-493.
[26] L. Fick, D. Blaauw, D. Sylvester, S. Skrzyniarz, M. Parikh, and D. Fick, "Analog inmemory subthreshold deep neural network accelerator," in Proc. IEEE Custom
Integrated Circuits Conf., Austin, TX, 2017,
pp. 1-4.
[27] Y. Lin, S. Zhang, and N. R. Shanbhag.
"Variation-tolerant architectures for convolutional neural networks in the near
threshold voltage regime," in Proc. IEEE
Int. Workshop Signal Processing Systems,
2016, pp. 17-22.
[28] P. Whatmough, S. Kyu Lee, H. Lee, S. Rama,
D. Brooks, and G.-Y. Wei, "A 28nm SoC
with a 1.2GHz 568nJ/pred sparse deep
neural network engine with >0.1 timing
error rate tolerance for IoT applications,"
in Proc. IEEE Int. Solid-State Circuits Conf.,
2017, pp. 242-243.
[29] G. Huang, et al. "Multi-scale dense convolutional networks for efficient prediction," arXiv Preprint, arXiv:1703.09844,
2017.
[30] J. Albericio, P. Judd, T. Hetherington, T.
Aamodt, N. E. Jerger, and A. Moshovos,
"Cnvlutin: Ineffectual-neuron-free deep
neural network computing," in Proc. ACM/
IEEE 43rd Annu. Int. Symp. Computer
Architecture, June 2016, pp. 1-13.
[31] D. Kim, J. Ahn, and S. Yoo, "A novel zero
weight/activation-aware hardware architecture of convolutional neural network,"
in Proc. IEEE Design, Automation & Test in
Europe Conf. & Exhibition, 2017, pp. 1462-
1467.
[32] S. Han, J. Pool, J. Tran, and W. Dally.
"Learning both weights and connections
for efficient neural network," in Proc. Advances in Neural Information Processing
Systems, 2015, pp. 1135-1143.
[33] V. Sze, T.-J. Yang, and Y.-H. Chen, "Designing energy-efficient convolutional neural
networks using energy-aware pruning,"
in Proc. Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, July
21-26, 2017, pp. 5687-5695.
[34] S. Han, H. Mao, and W. J. Dally, "Deep
compression: Compressing deep neural
networks with pruning, trained quantization and Huffman coding," arXiv Preprint,
arXiv:1510.00149, 2015.
[35] S. Han, X. Liu, H. Mao, J. Pu, A. Pedram,
M. A. Horowitz, and W. J. Dally, "EIE: Efficient inference engine on compressed
deep neural network," arXiv Preprint,
arXiv:1602.01528, 2016.
[36] J. Xue, J. Li, and Y. Gong, "Restructuring
of deep neural network acoustic models
with singular value decomposition," in
Proc. Interspeech Conf., 2013, pp. 2365-
2369.
[37] M. Verhelst. (2017). Deep learning processor survey. [Online]. Available: http://

www.esat.kuleuven.be/~mverhels/DLICsurvey.html
[38] S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, and
Y. Zou, "Dorefa-net: Training low bitwidth
convolutional neural networks with low
bitwidth gradients," arXiv preprint, arXiv:1606.06160.

About the Authors
Marian Verhelst (marian.verhelst@
kuleuven.be) has been an assistant
professor at the Micro-Electronics and
Sensors Laboratories of the Electrical
Engineering Department at KU Leuven, Belgium, since 2012. Her research
focuses on self-adaptive circuits and
systems, embedded machine learning, and low-power sensing and processing for the Internet of Things. She
received a Ph.D. degree from KU Leuven (cum ultima laude) in 2008. She
was a visiting scholar at the Berkeley Wireless Research Center of the
University of California, Berkeley, in
2005. From 2008 to 2011, she worked
in the Radio Integration Research
Lab of Intel Laboratories, Hillsboro,
Oregon. She is an IEEE Solid-State Circuits Society Distinguished Lecturer
and a member of the Young Academy
of Belgium and has published over
100 papers in conferences and journals. She is a member of the International Solid-State Circuits Conference
(ISSCC) Technical Program Committee
and the Design, Automation, and Test
in Europe (DATE) and ISSCC Executive
Committees. She was associate editor for IEEE Transactions on Circuits
and Systems II and currently serves in
the same capacity for IEEE Journal of
Solid-State Circuits.
Bert Moons received his B.S. and
M.S. degrees in electrical engineering
from KU Leuven, Belgium, in 2011 and
2013, respectively. In 2013, he joined
the Micro-Electronics and Sensors Laboratories of KU Leuven as a research
assistant, funded through an individual grant from the Research Foundation of Flanders. In 2016, he was a
visiting research student at Stanford
University, California, in the Murmann
Mixed-Signal Group. Currently, he is
working toward the Ph.D. degree on
energy-scalable and run-time adaptable digital circuits for embedded
deep learning applications.

IEEE SOLID-STATE CIRCUITS MAGAZINE

FA L L 2 0 17

65
http://www.esat.kuleuven.be/~mverhels/DLIC http://http://
Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Fall 2017
