IEEE Circuits and Systems Magazine - Q3 2023 - 38

output channel size is also known as the (weight) kernel
number.
The 2D convolutions can be viewed as drawing an
RS C×× cube inside the IA's HW C×× volume, and sliding
it across the IA's volume to obtain cubes of IA values.
For each cube, the dot-product between the cube with
each of the KR SC
×× kernels is performed to obtain
K OA values. Therefore, each 2D convolution can be formulated
as VMM with the RS C×× IA cube reshaped
to a vector, and the KR SC
×× kernels reshaped as a
matrix of K vectors.
C. Recurrent Neural Networks (RNNs)
An RNN and its more popular variants, gated recurrent
unit (GRU) and long short term memory (LSTM), are
used for sequence processing in speech recognition,
keyword detection, and natural language processing. An
RNN uses recurrent connections to process the input
sequence of the current timestep t and the output sequence
from the previous timestep t −1. An LSTM uses
input, output, forget gates, and a cell, i.e., if oc,, ,, to
keep track of features that are relevant in long term and
improves accuracy over traditional recurrent units. The
computation of an LSTM can also be formulated into a
VMM (Fig. 3(a)), where the input vectors are the input
sequence xt and the hidden sequence ht−1, the matrix
is the concatenation of if oc,, ,
matrices with respect to
the input or hidden sequences, and the output vector is
the hidden sequence ht .
D. Transformers
Recently, transformer architectures that use selfattention
and multi-head attention mechanisms are
getting increasingly better performance compared to
traditional LSTM in sequence and language applications
[5], [6], [7] and CNN in vision applications [4].
The multi-head attention computation is described in
Fig. 3(c). First, feed-forward operation, or matrix multiplication,
is applied on the input sequence to obtain
the key K(), query Q(), and value V() matrices with
its weights, WW
KQ
,, and WV, respectively. The KQ V,,
matrices are split into smaller matrices for multi-head
attention. Each attention block of the multi-head attention
performs the self-attention on its KQ V,, matrices.
The outputs from each attention block are concatenated,
then another feed-forward operation is applied to
obtain the final output sequence. The whole computation
can also be mapped into a series of matrix-matrix
multiplication (MMM).
III. Classic DNN Processing Architectures
and Dataflows
Single instruction multiple data (SIMD) and systolic array
are the basic architectures for computing VMM and
MMM. These two architectures and their variants form
the core of most of the DNN accelerators. In the following,
we review the two basic architectures and the common
dataflows for performing the computation.
Figure 3. Core computations of DNNs: (a) vector-matrix multiplication
in MLP and RNN, (b) 2D convolution in CNN, and
(c) multi-head attention in transformers.
38
IEEE CIRCUITS AND SYSTEMS MAGAZINE
A. SIMD Architecture
In general, a single instruction multiple data (SIMD) architecture
consists of an array of parallel processing
elements (PEs) or functional units (FUs) and performs
vector operations across an array of data. Only one instruction
is decoded and issued to trigger the computation
on multiple data across the array of PEs. Fig. 4(a)
illustrates the SIMD architecture for vector processing.
A SIMD array can be used to compute the dot-product
between two data vectors. Each PE receives a pair of
data from the memory or register file for multiplication,
then the result from each PE is written back to the memory
for the next summation instruction. Alternatively,
the results may be directly summed using an adder tree.
THIRD QUARTER 2023
IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023
