IEEE Solid-States Circuits Magazine - Summer 2020 - 33

For each of the following steps, certain constraints are added to reflect
changes in the assumptions on the
DNN processor or workload. The associated performance loss can, therefore,
be attributed to that change, and the final performance at one step becomes
the upper bound for the next step.
■■ Step 1 (layer shape and size): In
this first step, we look at the impact of the workload constraint so
that there is all spatial (i.e., parallel) processing and no temporal
(i.e., serial) processing. Therefore,
the performance upper bound is
determined by the finite size of
the workload (i.e., the number of
MAC operations in the layer).
■■ Step 2 (dataflow): In this step, we
specify the dataflow, which determines the order of operations and
where data are stored and reused,
and examine the impact of this
architectural constraint. Imposing a dataflow forces a serialization of processing and reduces
the performance upper bound,
which is the maximum parallelism of the dataflow.
■ ■ Step 3 (number of PEs) : In this
step, we restrict the system to a finite number of PEs and look at the
impact of this architectural constraint. A finite number of PEs can
degrade performance whenever
there is more parallel work to do
than that number of PEs. In addi-

(MAC/Cycle)

■■

tion, some of the PEs will be idle
(i.e., reducing the number of active
PEs) if the amount of work is not an
integer multiple of the number of
PEs (i.e., the work cannot be equally divided among the PEs).
Step 4 (physical dimensions of the
PE array): In this step, we consider the physical dimensions of
the PE array and data delivery
network (e.g., arranging 12 PEs as
3 # 4, 2 # 6, 4 # 3, and so on). The
spatial partitioning and associated on-chip network are often constrained per data type (e.g., input
activation or filter weight), which
can cause additional performance
loss because the required data
cannot be delivered to the PEs.
Step 5 (storage capacity): In this
step, we consider the impact of
making the buffer storage have finite capacity. Lack of storage can
limit parallelism when there is
insufficient storage to hold intermediate results and, thus, degrade
performance.
Step 6 (data BW): In this step, we
consider the impact of a finite BW
for delivering data across the different levels of the memory hierarchy.
The amount of data that needs to be
transferred between each level of
the memory hierarchy for each step
of computation and the available
data BW determine whether the PEs
can be kept busy.

Step 7 (varying data access patterns): In this step, we consider the
impact of BW varying across time
due to the dynamically changing
data access patterns (step 6 addresses only average BW). This
includes ramp-up time to initially
load values and ramp-down time
to drain values after completion.
Many common solutions are available to address this issue, including using double buffering, but
these can increase the area or reduce the amount of reuse.
Table 2 summarizes the constraints applied at each step of the
Eyexam process.
Up until this point, we have discussed how hardware design decisions
impact performance (i.e., throughput and latency). We now consider
how the choice of DNN model can
also have an effect. Specifically,
while the number of operations per
inference in (1) depends on the DNN
model, the number of operations per
second depends on both the DNN
model and the hardware. Thus, designing DNN models with efficient
layer shapes (also referred to as efficient network architectures, such as
MobileNet [17]) can reduce the number of MAC operations in the DNN
model and, consequently, the number
of operations per inference. However,
such DNN models can result in a wide
range of layer shapes, some of which
■■

Slope = BW to Only Active PE

Peak Performance

Step 1: Maximum Workload Parallelism
Step 2: Maximum Dataflow Parallelism
Number of PEs
Step 3: Number of Active PEs Under a Finite PE Array Size
Step 4: Number of Active PEs Under Fixed PE Array Dimensions
Step 5: Number of Active PEs Under Fixed Storage Capacity
Step 6: Lower Active PE Utilization Due to Insufficient Average BW
Step 7: Lower Active PE Utilization Due to Insufficient Instantaneous BW
(MAC/Data)
Workload
Operational Intensity

FIGURE 6: The impact of the Eyexam steps on the roofline model.

IEEE SOLID-STATE CIRCUITS MAGAZINE

SU M M E R 2 0 2 0

IEEE Solid-States Circuits Magazine - Summer 2020

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2020