IEEE Solid-States Circuits Magazine - Summer 2020 - 34

TABLE 2. A SUMMARY OF THE STEPS IN EYEXAM.
STEP

CONSTRAINT

TYPE

REASON FOR PERFORMANCE LOSS

1

Layer size and shape

Workload

Maximum workload parallelism

Finite workload size

2

Dataflow loop nest

Architectural

Maximum dataflow parallelism

Restricted dataflows defined by loop nest

3

Number of PEs

Architectural

Maximum PE parallelism

Additional restriction to mappings due to
shape fragmentation

4

Physical dimensions
of PE array

Architectural

Number of active PEs

Additional restriction to mappings due to
shape fragmentation for each dimension

5

Fixed storage
capacity

Architectural

Number of active PEs

Additional restriction to mappings due
to storage of intermediate data (depends
on dataflow)

6

Fixed data BW

Microarchitectural

Maximum data BW to active PEs

Insufficient average BW to active PEs

7

Varying data access
patterns

Microarchitectural

Actual measured performance

Insufficient instant BW to active PEs

may have poor utilization of PEs, and
thus reduce the overall operations
per second, as shown in (2).
A deeper consideration of the operations per second is that all operations
are not created equal, so cycles per
operation may not be constant. For
example, if we consider the fact that
anything multiplied by zero is zero,
some MAC operations are ineffectual
(i.e., they do not change the accumulated value). The number of ineffectual operations is a function of both
the weights in the DNN model and the
input data. These ineffectual MAC operations can require fewer cycles or
no cycles at all. Conversely, we need
to process only effectual (or nonzero)
MAC operations, where both inputs
are nonzero; this is referred to as exploiting sparsity. Various hardware
architectures have been proposed to
exploit sparsity [18]-[20].
Processing only effectual MAC
operations can increase the (total) operations per second by increasing the
(total) operations per cycle. (By total
operations, we mean both effectual
and ineffectual operations.) Ideally,
the hardware would skip all ineffectual
operations; however, in practice, designing hardware to skip all ineffectual
operations can be challenging and result in increased hardware complexity
and overhead. For instance, it might be
easier to design hardware that recognizes zeros in only one of the operands
(e.g., weights) rather than both. There-

34	

NEW PERFORMANCE BOUND

SU M M E R 2 0 2 0	

fore, the ineffectual operations can be
further divided into those that are exploited by the hardware (i.e., skipped)
and those that are unexploited by the
hardware (i.e., not skipped). The number of operations actually performed
by the hardware is, therefore, the effectual operations plus unexploited
ineffectual operations.
Equation (4) at the bottom of the
page shows how operations per cycle
can be decomposed into
■■ the number of effectual operations
plus unexploited ineffectual operations per cycle, which remains
somewhat constant for a given
hardware architecture design
■■ the ratio of effectual operations
over effectual operations plus unexploited ineffectual operations,
which refers to the ability of the
hardware to exploit ineffectual operations (ideally, unexploited ineffectual operations should be zero,
and this ratio should be one)
■■ the number of effectual operations
out of (total) operations, which is
related to the amount of sparsity
and depends on the DNN model.

As the amount of sparsity increases [i.e.,
the number of effectual operations
out of (total) operations decreases],
the operations per cycle increases, as
shown in (4); this subsequently increases
operations per second, as shown in (2):
However, exploiting sparsity requires additional hardware to identify
when inputs to the MAC are zero to
avoid performing unnecessary MAC
operations. The additional hardware
can increase the critical path, which
decreases cycles per second, and can
also increase the area of the PE, which
reduces the number of PEs for a given
area. Both of these factors can reduce
the operations per second, as shown
in (2). Therefore, the complexity of
the additional hardware can result in
a tradeoff between reducing the number of unexploited ineffectual operations and increasing the critical path
or reducing the number of PEs.
Finally, designing hardware and
DNN models that support reduced
precision (i.e., fewer bits per operand
and per operation) can also increase
the number of operations per second.
Fewer bits per operand means that

operations effectual operations + unexploited ineffectual operations
=
cycle
cycle
effectual operations
#
effectual operations + unexploited ineffectual operations

1
(4)
#
.
effectual operations
operations

IEEE SOLID-STATE CIRCUITS MAGAZINE
IEEE Solid-States Circuits Magazine - Summer 2020

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2020
