IEEE Circuits and Systems Magazine - Q3 2023 - 52

channels, or a total bandwidth of 640 Gbps, to communicate
with the FPGA chiplet [55].
With heterogeneous integration, the FPGA can serve
as the flexible platform that can be configured to serve
as the host and handle control and data management,
and the PETRA systolic array chiplet can perform the
VMM and MMM that form the core part of DNN computation
[55]. Operations that are not supported by the PETRA
chiplet can always be covered by the FPGA chiplet.
The heterogeneous platform can be further extended,
e.g., by adding a front-end chiplet to make a complete
sensor platform, and by adding another function accelerator
chiplet to expand the capability of the system.
VI. Conclusion
DNN hardware design is a fast-evolving field. In this article
we provide a survey and a tutorial on the basics of
the DNN workloads, the essential processing architectures,
and the promising directions in sparse DNN processing
and multi-chip integration. First, we explain the
two basic architectures for DNN processing, SIMD and
systolic array, along with common WS and OS dataflows,
to show the tradeoffs between flexibility and energy efficiency,
and utilization and compute density. Next, we
present designs that exploit data sparsity to improve
both performance and energy efficiency with compressed
storage and sparse processing. From partial
sparsity to full sparsity, architectures can be designed
with a range of overheads to gain from an array of benefits
including lower energy, smaller memory, lower
memory bandwidth and higher performance. Lastly, we
show a path in scaling up and scaling out DNN hardware
using multi-chiplet integration, either by tiling of modular
DNN chiplets in constructing larger-scale systems or
by heterogeneously pairing of DNN chiplets with CPU or
FPGA to build a versatile platform.
Acknowledgment
This work was supported in part by ACE, one of the seven
centers in JUMP 2.0, a Semiconductor Research Corporation
(SRC) Program sponsored by DARPA.
Jie-Fang Zhang (Member, IEEE) received
the B.S. degree in electrical engineering
from National Taiwan University,
Taipei, Taiwan, in 2015, and the
M.S. degree in computer science and
engineering and the Ph.D. degree in
electrical and computer engineering from the University
of Michigan, Ann Arbor, MI, USA, in 2018 and 2022, respectively.
He joined NVIDIA in 2022 as a Deep Learning
52
IEEE CIRCUITS AND SYSTEMS MAGAZINE
Architect focusing on GPU performance analysis, modeling,
and optimization for deep learning models. His research
interests include energy-efficient hardware architecture
and accelerator design for machine learning,
computer vision, and robotics applications.
Zhengya
Zhang (Senior Member,
IEEE) received the B.A.Sc. degree in
computer engineering from the University
of Waterloo in 2003, and the M.S.
and Ph.D. degrees in electrical engineering
from the University of California
at Berkeley (UC Berkeley), Berkeley, CA, USA, in 2005
and 2009, respectively. He has been a Faculty Member
with the University of Michigan, Ann Arbor, MI, USA,
since 2009, where he is currently a Professor with the
Department of Electrical Engineering and Computer Science.
His research interests include low-power and highperformance
VLSI circuits and systems for computing,
communications, and signal processing. He was a recipient
of the University of Michigan College of Engineering
Neil Van Eenam Memorial Award in 2019, the Intel Early
Career Faculty Award in 2013, the National Science Foundation
CAREER Award in 2011, and the David J. Sakrison
Memorial Prize from UC Berkeley in 2009. He has been
an Associate Editor of the IEEE TransacTions on Very Large
scaLe inTegraTion (VLsi) sysTems since 2015, and serves on
the Technical Program Committee of the IEEE Custom
Integrated Circuits Conference (CICC) since 2018. He was
an Associate Editor of the ieee TransacTions on circuiTs
and sysTems-ParT i: reguLar PaPers from 2013 to 2015 and
the IEEE TransacTions on circuiTs and sysTems-ParT ii: exPress
Briefs from 2014 to 2015, and served on the Technical
Program Committee of the ieee VLsi Symposium on
Technology and Circuits from 2018 to 2022. He is an IEEE
Solid-State Circuits Society Distinguished Lecturer.
References
[1] Y. LeCun, Y. Bengio, and G. Hinton, " Deep learning, " Nature, vol. 521,
no. 7553, pp. 436-444, May 2015.
[2] K. He et al., " Deep residual learning for image recognition, " in Proc.
Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770-778.
[3] C. Szegedy et al., " Rethinking the inception architecture for computer
vision, " in Proc. the Conf. Comput. Vis. Pattern Recognit. (CVPR),
Jun. 2016, pp. 2818-2826.
[4] A. Dosovitskiy et al., " An image is worth 16x16 words: Transformers
for image recognition at scale, " 2020, arXiv:2010.11929.
[5] A. Vaswani et al., " Attention is all you need, " in Proc. Adv. Neural Inf.
Process. Syst. (NIPS), 2017, pp. 6000-6010.
[6] J. Devlin et al., " BERT: Pretraining of deep bidirectional transformers
for language understanding, " in Proc. Conf. North Amer. Chapter Assoc.
Comput. Linguistics, Human Language Technol., vol. 1, Jun. 2019, Art.
no. 41714186.
[7] T. Brown et al., " Language models are few-shot learners, " in Proc.
Adv. Neural Inf. Process. Syst. (NIPS), 2020, pp. 1877-1901.
[8] A. Canziani, A. Paszke, and E. Culurciello, " An analysis of deep neural
network models for practical applications, " 2017, arXiv:1605.07678.
THIRD QUARTER 2023
IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023
