IEEE Solid-States Circuits Magazine - Summer 2020 - 38

Flexibility
Problem
Specification/Shape
(DNN Model)

Architecture
and
Implementation
Description

Timeloop
(DNN Mapping Tool:
Map Space Creation,
Search, Performance Model)

Action
Counts

Accelergy
(Energy Estimator Tool)

Energy
Estimation
Plug-In 0

Energy
Estimation
Plug-In 1

Energy
Estimation

...

FIGURE 9: Timeloop [13] with the integration of Accelergy [34] as an energy estimation
model. Timeloop sends projected action counts for a mapping to Accelergy and receives an
energy estimation to guide its search. Accelergy plug-ins allow for the customization of component energy estimation. These tools are available at http://accelergy.mit.edu/tutorial.html.

From an industry perspective, the
cost constraints are related to volume
and market; for instance, embedded
processors have much more stringent cost limitations than processors
in the cloud. One of the key factors
that affects cost is the chip area (e.g.,
square millimeters) in conjunction
with the process technology (e.g.,
45-nm CMOS), which constrains the
amount of on-chip storage as well as
amount of compute (e.g., the number
of PEs for DNN processors, the number of cores for CPUs and GPUs, the
number of digital signal processing
(DSP) engines for field-programmable gate arrays (FPGAs), and so on).
To report information related to area
without specifying a specific process technology, one can report the
amount of on-chip memory (e.g., the
storage capacity of the global buffer)
and compute (e.g., the number of PEs)
as a proxy for area.
Another important factor is the
amount of off-chip BW, which dictates the cost and complexity of the
packaging and printed circuit board
(PCB) design [e.g., High Bandwidth
Memory (HBM) [31] to connect to offchip DRAM, NVLink [38] to connect
to other GPUs, and so on] as well as
whether additional chip area is required for a transceiver to handle

SU M M E R 2 0 2 0

signal integrity at high speeds. The
off-chip BW, which is typically reported in gigabits per second and
the number of ports, can be used as
a proxy for packaging and PCB cost.
There is also an interplay between
the costs attributable to the chip
area and off-chip BW. For instance,
increasing on-chip storage, which increases chip area, can reduce off-chip
BW. Accordingly, both metrics should
be reported to provide perspective on
the total cost of the system.
Of course, reducing cost alone is
not the only objective. The design
objective is, invariably, to maximize
the throughput or energy efficiency
for a given cost, specifically to maximize inferences per second per cost
(e.g., U.S. dollars) and/or inferences
per joule per cost. This is closely related to the previously discussed property of utilization; to be cost efficient,
the design should aim to utilize every
PE to increase inferences per second,
since each PE increases the area and,
thus, the cost of the chip; similarly,
the design should aim to effectively
utilize all of the on-chip storage to
reduce off-chip BW or increase operations per off-chip memory access
as expressed by the roofline model
(see Figure 5), as each byte of on-chip
memory also increases cost.

IEEE SOLID-STATE CIRCUITS MAGAZINE

The merit of a DNN processor is also
a function of its flexibility, which
refers to the range of DNN models
that can be supported on the DNN
processor and the ability of the software environment (e.g., the mapper)
to maximally exploit the capabilities
of the hardware for any desired DNN
model. Given the fast-moving pace
of DNN research and deployment, it
is increasingly important that DNN
processors s
- upport a wide range of
DNN models and tasks.
We can define support in two tiers.
The first tier requires only that the
hardware needs to be able to functionally support different DNN models (i.e., the DNN model can run on the
hardware). The second tier requires
that the hardware also maintain efficiency (i.e., high throughput and
energy efficiency) across different
DNN models.
To maintain efficiency, the hardware should not rely on certain properties of the DNN models to achieve
efficiency, as the properties of DNN
models are diverse and evolving
rapidly. For instance, a DNN processor that can efficiently support the
case where the entire DNN model
(i.e., all of the weights) fits on chip
may perform extremely poorly when
the DNN model grows larger, which
is likely, given that the size of DNN
models continues to increase over
time; a more flexible processor would
be able to efficiently handle a wide
range of DNN models, even those that
exceed on-chip memory.
The degree of flexibility provided
by a DNN processor presents a complex tradeoff with processor cost.
Specifically, additional hardware
usually needs to be added to flexibly
support a wider range of workloads
and/or improve their throughput
and energ y efficiency. Thus, the
design objective is to reduce the
overhead (e.g., area cost and energy
consumption) of supporting flexibility while maintaining efficiency
across the wide range of DNN models. Therefore, evaluating flexibility
would entail ensuring that the extra

http://accelergy.mit.edu/tutorial.html

IEEE Solid-States Circuits Magazine - Summer 2020

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2020