IEEE Computational Intelligence Magazine - August 2021 - 12

Dataflow design is one of the most important features
in hardware accelerators, which allows the system to
reuse the data among different processing elements.
weight parameters are zeros, the computation process can be
skipped in those PEs, thus reducing energy consumption.
Quantization is another critical model compression technique
that is used to accelerate DNNs and reduce model size
[10]. It involves mapping data to a small set of quantization levels
and aims at minimizing the error between the reconstructed
data from the quantization levels and the original data. The
quantization level reflects the precision and ultimately the
number of bits representing a parameter. After the quantization,
the low precision parameters may still store enough information
for model inference, and the accuracy of the model can be
maintained. In practical implementations, if the weights are
quantized, one can use multipliers with simpler structures, thus
reducing energy consumption. For instance, a high precision
parameter with 32-bit float point (32FP) data type requires
23 bit × 23 bit multipliers. Such type of multiplier contains
506 adders in total. If quantizing the activation from 32FP to
16-bit float point (16FP) and quantizing the weights from
32FP to 8-bit integer (8INT), only 10 bit × 8 bit multipliers
are required, each of which contains only 72 adders in total.
The fewer adders in multipliers, the lower energy consumption
for computation.
Pruning and quantization can reduce not only the energy
consumption on the computation process but also the energy
consumption on the data movement, which is roughly proportional
to the total amount of data transmitted from the memory
module in terms of bits [15]. For instance, if pruning 80% of
the parameters in the model and quantizing all the parameters
from 16 bits to 8 bits, then about 90% of the energy consumption
on data movement can be reduced.
B. Dataflow Design in Hardware Accelerators
The dataflow design decides how data is reused among different
PEs. Since a large portion of the energy consumption of
hardware accelerators is on the data movement, the dataflow
design needs to be considered when optimizing the energy
efficiency. Algorithm 1 shows the computation procedure of a
Algorithm 1 Computation of a typical convolutional layer.
for co in range(CO) do
for ci in range(CI) do
for x in range(X) do
for y in range(Y) do
for fx from -(FX-1)/2 to (FX-1)/2 do
for fy from -(FY-1)/2 to (FY-1)/2 do
O[co][x][y]
+ = I[ci][x + fx][y + fy] W# [co][ci][fx][fy]
map, and FX
typical convolutional layer. It contains six
loops, each of which corresponds to one
dimension either in the weight filter or in the
feature map. More specifically, CO
and CI
and FY
are
the numbers of output and input channels, X
and Y are the width and height of the feature
are the width and height of the weight
filter. In each iteration of the innermost loop, a basic arithmetic
operation called multiply-accumulate (MAC) is performed. In
one convolutional layer, there are ·· ·· ·CC XYFFOI XY
MAC
operations in total.
In typical hardware accelerators,3 there are a set of processing
elements. Each PE can execute one MAC operation independently.
How to map the MAC operation into each PE and
how the data flow among those PEs become key considerations
in the design of hardware accelerators. Theoretically,
there are many mapping methods, resulting in different dataflow
designs. For example, suppose that the device has an array
of PEs, one can unroll any one of the loops in Algorithm 1
and map each iteration in the unrolled loop into each PE of
the array. Similarly, if the device has a matrix of PEs, one can
unroll any two loops in Algorithm 1 and map the MAC operations
into each PE in the matrix. Thus, with six loops as
shown in Algorithm 1, there are C 156
2 = possible dataflow
designs in total. To simplify the problem, in this work, only four
popular dataflow designs are evaluated, as shown in Table I.
These dataflow designs are named as : ,AB where A and B
stand for the names of the two unrolled loops.
Figure 1 shows the schematic diagram of those four popular
dataflow designs, where only four PEs are involved in each
example. Each PE contains one multiplier and one adder,
which can execute one MAC operation each time. The PE also
contains register files, which can temporarily store input or
output data. In X:Y, the MAC operation results are stored in
registers at output ports of PEs. At each iteration, the last MAC
operation result is read from registers. In :,
tion, the input feature map is reused CO
operation results are summed up. In :,
CC at each iteratimes,
and CI
IO
MAC
FF FF ·XY weights are
XY
stored in registers at input ports of PEs. At each iteration,
FF ·XY
MAC operation results are summed up. In :,
XF FXX
weights are stored in registers at input ports of PEs. At each
iteration, the weights are reused X times, and FX
tion results are summed up.
TABLE I Popular dataflow designs
that are applied in literature.
DATAFLOW
XY:
FF :XY
APPLIED BY
[16] [17]
[20]
DATAFLOW
CC :IO
XF : X
APPLIED BY
[18] [19]
[21] [22]
MAC opera3The
devices that are specialized to execute a certain task, such as the graphics processing
unit (GPU), field-programmable gate array(FPGA), and application-specific
integrated circuit (ASIC).
12 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2021
IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
