IEEE Circuits and Systems Magazine - Q3 2023 - 37

for DNN workloads is of importance to answer new application
needs. A prime example of a domain-specific
accelerator for DNN is Google's TPU [13].
Designing DNN-based ML accelerators has been a
rapid-growing field. In general, we identify three major
challenges that need to be addressed in designing these
ML accelerators.
■ Flexibility: The design needs to be flexible to support
a variety of computation types and models in
the DNN workload class, not only for the current
generation of DNN models, but also for future generations
as the models are evolving more quickly
than hardware upgrades.
■ Efficiency: The design needs to optimize both the
processing and the memory access to provide a
competitive advantage over GPUs and CPUs and
answer new application needs.
■ Scalability: The design needs to provide a way to
support larger models with higher memory and
computation requirements, and new variations of
the current models to remain relevant.
In this review article, we discuss three important directions
to address the computation challenges in supporting
modern ML models and workloads. First, we
describe the common processing architectures and the
data reuse opportunities for ML computation. Then, we
present the benefit of exploiting data-level sparsity to
improve computation efficiency. Lastly, we provide an
overview of scaling-up and scaling-out approaches to
answer the scalability challenge.
This article is organized as follows. In Section II, we
present the primary types of computation used in ML
and DNN workloads. We then describe two common
processing architectures for DNN computation and
common stationary dataflows to exploit data reuse in
Section III. To gain better performance and efficiency, an
effective approach is by exploiting data sparsity, which
is explained in Section IV along with examples of sparse
compression formats and sparse architectures. To scale
up designs and scale out its functionalities, a chipletbased
approach can be effectively employed. We review
examples of homogeneous tiling and heterogeneous integration
of chiplets in Section V to demonstrate promising
recent results. Finally, we conclude this article in
Section VI.
II. Background
In general, we can broadly categorize DNN models into
four types based on its network structure and computation:
1) multi-layer perceptron (MLP), 2) convolutional
neural network (CNN), 3) recurrent neural network
(RNN), and 4) transformer. Here, we present the
THIRD QUARTER 2023
high-level structures of each model and explain its core
computation.
A. Multi-Layer Perceptron (MLP)
An MLP consists of multiple feedforward fully-connected
(FC) layers cascaded one after another. The computation
of an FC layer can be formulated into a vectormatrix
multiplication (VMM) between the input vector
x
∈ and the weight matrix W∈ ×KC to obtain the
output vector y∈K, as described in Fig. 3(a).
C
B. Convolutional Neural Networks (CNNs)
CNNs are mostly specialized for 2D image processing
in vision applications, e.g., image classification, object
detection, and semantic segmentation [2], [3], [4], [12],
[14], [15], [16], [17], [18]. A CNN uses convolution (CONV)
layers for spatial feature extraction and FC layers for
feature classification. The input and output are often
referred as input activation (IA) and output activation
(OA). A CONV layer has a weight (W) of size RS CK×× × ,
which can be understood as K 3D kernels of RS C×× .
A CONV layer processing takes an IA of size HW C××
and performs 2D convolutions between the IA and the K
3D kernels to obtain an OA of size HW K×× , as shown
in Fig. 3(b). The model hyperparameters C and K are
the input and output channel sizes, respectively. The
Figure 2. Evolution of model size in the fields of (a) CV and
(b) NLP. Adapted from [11].
IEEE CIRCUITS AND SYSTEMS MAGAZINE
37
IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023
