IEEE Solid-States Circuits Magazine - Summer 2022 - 77

consume low leakage. Also, with the
resistive device, it can naturally support
matrix-vector multiplication.
On the other hand, the nonidealities
of eNVM devices include endurance,
variation, drift, and so on. In several
recent works, the larger-scale integration
of eNVM-based IMC systems
has been demonstrated. In [63], an
embedded Flash-based IMC system
with 79 million 8-b weights was presented,
and [64] presented a 2.25-MB
RRAM-based IMC system with an
embedded ARM processor.
Addressing Pruning for IMC Designs
Applying random sparsity patterns
resulted from fine-grain nonstructured
pruning to a fixed SRAM IMC
array structure can become inefficient
[65]. If the IMC operation happens
on a column basis, it will be
much more efficient to prune out
the entire/partial column in a structured
manner. However, as shown
in Figure 10, smaller group sizes
achieve higher sparsity compared
to the large-sized groups. On the
other hand, a small-sized group will
restrict the number of rows that can
be activated simultaneously, which
requires a higher number of cycles to
go through the same crossbar array.
Other approaches include sparsityaware
activation/weight processing
for IMC macro design [16] and
sparsity-optimized IMC bitcell design
[66]. Overall, this challenge needs a
carefully structured pruning algorithm
and supporting IMC hardware
co-design.
Conclusion
In this article, we presented how
both digital and analog AI accelerators
have largely advanced in recent
years. The trends of all-digital accelerators
include a reconfigurable MAC
array, high utilization across various
AI models, flexible precision support,
and weight/activation sparsityaware
design. Regarding the trends
on analog/mixed-signal accelerators,
single IMC macro designs are scaled
up with a higher level of integration
for a many-macro IMC system design.
Flexible precision and programmability
have been supported in
larger-scale IMC systems. The challenges
for these designs are being
addressed in different ways. For analog
IMC designs, improving the DNN
accuracy, ADC overhead, density, and
sparsity are important. By addressing
such challenges, new all-analog,
digital IMC- and NVM-based AI accelerators
are being further presented
in the literature.
Acknowledgments
This work was, in part, supported
by National Science Foundation grant
1652866 and the Center for BrainInspired
Computing, one of six
centers in the Joint University Microelectronics
Program, a Semiconductor
Research Corporation program sponsored
by DARPA.
References
[1] L. Deng, G. Li, S. Han, L. Shi, and Y. Xie,
" Model compression and hardware acceleration
for neural networks: A comprehensive
survey, " Proc. IEEE, vol. 108,
no. 4, pp. 485-532, 2020, doi: 10.1109/
JPROC.2020.2976475.
[2] G. W. Burr, S. Lim, B. Murmann, R. Venkatesan,
and M. Verhelst, " Fair and comprehensive
benchmarking
learning
processing
of machine
chips, " IEEE Des.
Test, vol. 39, no. 3, pp. 18-27, 2022, doi:
10.1109/MDAT.2021.3063366.
[3] J. Choi, S. Venkataramani, V. V. Srinivasan,
K. Gopalakrishnan, Z. Wang, and P. Chuang,
" Accurate and efficient 2-bit quantized neural
networks, " in Proc. Conf. Mach. Learn.
Syst. (MLSys), 2019, pp. 348-359.
[4] S. Han, H. Mao, and W. J. Dally,
" Deep
compression: Compressing deep neural
networks with pruning, trained quantization
and Huffman coding, " in Proc. Int.
Conf. Learn. Representations (ICLR), 2016,
pp. 1-14.
[5] H. Cai, T. Chen, W. Zhang, Y. Yu, and J.
Wang, " Efficient architecture search by
network transformation, " in Proc. AAAI
Conf. Artif. Intell., 2018, pp. 2787-2794,
doi: 10.5555/3504035.3504375.
[6] A. Novikov, D. Podoprikhin, A. Osokin,
and D. P. Vetrov, " Tensorizing Neural Networks, "
in Proc. Adv. Neural Inf. Process.
Syst. (NeurIPS), 2015, vol. 28, pp. 442-450,
doi: 10.5555/2969239.2969289.
[7] D. Shin, J. Lee, J. Lee, and H.-J. Yoo, " DNPU:
An 8.1TOPS/W reconfigurable CNN-RNN
processor for general-purpose deep neural
networks, " in Proc. IEEE Int. Solid-State
Circuits Conf. (ISSCC), 2017, pp. 240-241,
doi: 10.1109/ISSCC.2017.7870350.
[8] K. Wang, Z. Liu, Y. Lin, J. Lin, and S.
Han, " HAQ: Hardware-aware automated
quantization with mixed precision, " in
Proc. IEEE/CVF Conf. Comput. Vis. Pattern
Recognit. (CVPR), 2019, pp. 8612-8620.
[9] I. Hubara, M. Courbariaux, D. Soudry, R.
El-Yaniv, and Y. Bengio, " Quantized neural
networks: Training neural networks with
low precision weights and activations, " J.
Mach. Learn. Res., vol. 18, no. 1, pp. 6869-
6898, 2018.
[10] M. Rastegari, V. Ordonez, J. Redmon, and
A. Farhadi, " XNOR-Net:
sification
ImageNet
using
binary
clasconvolutional
neural
networks, " in Proc. Eur. Conf. Comput.
Vis. (ECCV), 2016, pp. 525-542, doi:
10.1007/978-3-319-46493-0_32.
[11] A. Mishra, E. Nurvitadhi, J. J. Cook, and
D. Marra, " WRPN: Wide reduced-precision
networks, " in Proc. Int. Conf. Learn. Representations
(ICLR), 2018, pp. 1-11.
[12] S. K. Esser, J. L. McKinstry, D. Bablani, R.
Appuswamy, and D. S. Modha, " Learned
step size quantization, " in Proc. Int. Conf.
Learning Representations (ICLR), 2020, pp.
1-12.
[13] E. Park and S. Yoo,
" PROFIT: A novel
training method for sub-4-bit MobileNet
models, " in Proc. Eur. Conf. Comput.
Vis.
(ECCV), 2020, pp. 430-446, doi:
10.1007/978-3-030-58539-6_26.
[14] S. Park, J. Lee, S. Mo, and J. Shin, " Lookahead:
A far-sighted alternative of magnitude-based
pruning, " in Proc. Int. Conf.
Learn. Representations (ICLR), 2020, pp.
1-20.
[15] W. Wen, C. Wu, Y. Wang, Y. Chen, and H.
Li, " Learning structured sparsity in deep
neural networks, " in Proc. Adv. Neural Inf.
Process. Syst. (NeurIPS), 2016, pp. 2074-
2082, doi: 10.5555/3157096.3157329.
[16] J. Yue et al., " A 65nm computing-inmemory-based
CNN processor with
2.9-to-35.8TOPS/W system energy efficiency
using dynamic-sparsity performance-scaling
architecture and energyefficient
inter/intra-macro data reuse, "
in Proc. IEEE Int. Solid- State Circuits Conf.
(ISSCC), 2020, pp. 234-236, doi: 10.1109/
ISSCC19947.2020.9062958.
[17] D. Kadetotad, S. Yin, V. Berisha, C.
Chakrabarti, and J. Seo, " An 8.93 TOPS/W
LSTM recurrent neural network accelerator
featuring hierarchical coarse-grain
sparsity for on-device speech recognition, "
IEEE J. Solid-State Circuits, vol. 55,
no. 7, pp. 1877-1887, 2020, doi: 10.1109/
JSSC.2020.2992900.
[18] L. Yang, Z. He, and D. Fan, " Harmonious
coexistence of structured weight pruning
and ternarization for deep neural networks, "
in Proc. AAAI Conf. Artif. Intell.,
vol. 34, no. 4, pp. 6623-6630, 2020, doi:
10.1609/aaai.v34i04.6138.
[19] N. P. Jouppi et al., " In-datacenter performance
analysis of a tensor processing
unit, " in Proc. ACM/IEEE 44th Annu. Int.
Symp. Comput. Architecture (ISCA), 2017,
pp. 1-12, doi: 10.1145/3079856.3080246.
[20] J.-S. Park et al., " A 6K-MAC feature-mapsparsity-aware
neural processing unit
in 5nm flagship mobile SoC, " in Proc. IEEE
Int. Solid-State Circuits Conf., 2021, vol. 64,
pp. 152-154, doi: 10.1109/ISSCC42613.2021.
9365928.
[21] B. Moons, R. Uytterhoeven, W. Dehaene,
and M. Verhelst, " ENVISION: A 0.26-to-10
TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable
convolutional
neural network processor in 28nm
FDSOI, " in Proc. IEEE Int. Solid-State Circuits
Conf. (ISSCC), 2017, pp. 246-247, doi:
10.1109/ISSCC.2017.7870353.
[22] J. Lee, C. Kim, S. Kang, D. Shin, S. Kim,
and H.-J. Yoo, " UNPU: An energy-efficient
deep neural network accelerator
with fully variable weight bit precision, "
IEEE J. Solid-State Circuits, vol. 54, no.
1, pp. 173-185, 2019, doi: 10.1109/JSSC.
2018.2865489.
[23] P. C. Knag et al., " A 617-TOPS/W all-digital
binary neural network accelerator in 10nm
FinFET CMOS, " IEEE J. Solid-State
IEEE SOLID-STATE CIRCUITS MAGAZINE
SUMMER 2022
77
IEEE Solid-States Circuits Magazine - Summer 2022

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2022
