IEEE Solid-States Circuits Magazine - Summer 2020 - 28

ISSCC 2020 Tutorial
Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S. Emer

How to
Evaluate Deep Neural
Network Processors
TOPS/W (alone) Considered Harmful

A

significant amount
of specia lized
hardware has been
developed for processing deep neural
networks (DNNs) in both academia and
industry. This article aims to highlight
the key concepts required to evaluate
and compare these DNN processors.
We discuss existing challenges, such
as the flexibility and scalability needed to support a wide range of neural
networks, as well as design considerations for both the DNN processors
and the DNN models themselves. We
also describe specific metrics that can
be used to evaluate and compare existing solutions beyond the commonly
used tera-operations per second per
watt (TOPS/W). This article is based on
the tutorial "How to Understand and
Evaluate Deep Learning Processors"
that was given at the 2020 InternaDigital Object Identifier 10.1109/MSSC.2020.3002140
Date of current version: 25 August 2020

28

SU M M E R 2 0 2 0

tional Solid-State Circuits Conference,
as well as excerpts from the book, Efficient Processing of Deep Neural Networks [36].

Motivation and Background
Over the past few years, there has
been a significant amount of research
on enabling the efficient processing
of DNNs. The challenge of efficient
DNN processing depends on balancing multiple objectives:
■ high performance (including accuracy) and efficiency (including
cost)
■ enough flexibility to cater to a
wide and rapidly changing range
of workloads
■ good integration with existing
software frameworks.
DNN computations are composed
of several processing layers (Figure  1), where, for many layers, the
main computation is a weighted sum;
in other words, the main computation for DNN processing is often a

IEEE SOLID-STATE CIRCUITS MAGAZINE

multiply-accumulate (MAC) operation. The arrangement of the MAC
operations within a layer is defined
by the layer shape; for instance,
Table 1 and Figure 2 highlight the
shape parameters for layers used
in convolutional neural networks
(CNNs), a popular type of DNN. Because the shape parameters can vary
across layers, DNNs come in a wide
variety of shapes and sizes, depending on the application. (The DNN research community often refers to the
shape and size of a DNN as its network architecture. However, to avoid
confusion with the use of the word
architecture by the hardware community, we talk about DNN models
and their shape and size in this article.) This variety is one of the motivations for flexibility, and it causes
the objectives listed previously to be
highly interrelated.
Figure 3 illustrates the hardware
architecture of a typical DNN processor, which is composed of an array

1943-0582/20©2020IEEE



IEEE Solid-States Circuits Magazine - Summer 2020

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2020

Contents
IEEE Solid-States Circuits Magazine - Summer 2020 - Cover1
IEEE Solid-States Circuits Magazine - Summer 2020 - Cover2
IEEE Solid-States Circuits Magazine - Summer 2020 - Contents
IEEE Solid-States Circuits Magazine - Summer 2020 - 2
IEEE Solid-States Circuits Magazine - Summer 2020 - 3
IEEE Solid-States Circuits Magazine - Summer 2020 - 4
IEEE Solid-States Circuits Magazine - Summer 2020 - 5
IEEE Solid-States Circuits Magazine - Summer 2020 - 6
IEEE Solid-States Circuits Magazine - Summer 2020 - 7
IEEE Solid-States Circuits Magazine - Summer 2020 - 8
IEEE Solid-States Circuits Magazine - Summer 2020 - 9
IEEE Solid-States Circuits Magazine - Summer 2020 - 10
IEEE Solid-States Circuits Magazine - Summer 2020 - 11
IEEE Solid-States Circuits Magazine - Summer 2020 - 12
IEEE Solid-States Circuits Magazine - Summer 2020 - 13
IEEE Solid-States Circuits Magazine - Summer 2020 - 14
IEEE Solid-States Circuits Magazine - Summer 2020 - 15
IEEE Solid-States Circuits Magazine - Summer 2020 - 16
IEEE Solid-States Circuits Magazine - Summer 2020 - 17
IEEE Solid-States Circuits Magazine - Summer 2020 - 18
IEEE Solid-States Circuits Magazine - Summer 2020 - 19
IEEE Solid-States Circuits Magazine - Summer 2020 - 20
IEEE Solid-States Circuits Magazine - Summer 2020 - 21
IEEE Solid-States Circuits Magazine - Summer 2020 - 22
IEEE Solid-States Circuits Magazine - Summer 2020 - 23
IEEE Solid-States Circuits Magazine - Summer 2020 - 24
IEEE Solid-States Circuits Magazine - Summer 2020 - 25
IEEE Solid-States Circuits Magazine - Summer 2020 - 26
IEEE Solid-States Circuits Magazine - Summer 2020 - 27
IEEE Solid-States Circuits Magazine - Summer 2020 - 28
IEEE Solid-States Circuits Magazine - Summer 2020 - 29
IEEE Solid-States Circuits Magazine - Summer 2020 - 30
IEEE Solid-States Circuits Magazine - Summer 2020 - 31
IEEE Solid-States Circuits Magazine - Summer 2020 - 32
IEEE Solid-States Circuits Magazine - Summer 2020 - 33
IEEE Solid-States Circuits Magazine - Summer 2020 - 34
IEEE Solid-States Circuits Magazine - Summer 2020 - 35
IEEE Solid-States Circuits Magazine - Summer 2020 - 36
IEEE Solid-States Circuits Magazine - Summer 2020 - 37
IEEE Solid-States Circuits Magazine - Summer 2020 - 38
IEEE Solid-States Circuits Magazine - Summer 2020 - 39
IEEE Solid-States Circuits Magazine - Summer 2020 - 40
IEEE Solid-States Circuits Magazine - Summer 2020 - 41
IEEE Solid-States Circuits Magazine - Summer 2020 - 42
IEEE Solid-States Circuits Magazine - Summer 2020 - 43
IEEE Solid-States Circuits Magazine - Summer 2020 - 44
IEEE Solid-States Circuits Magazine - Summer 2020 - 45
IEEE Solid-States Circuits Magazine - Summer 2020 - 46
IEEE Solid-States Circuits Magazine - Summer 2020 - 47
IEEE Solid-States Circuits Magazine - Summer 2020 - 48
IEEE Solid-States Circuits Magazine - Summer 2020 - 49
IEEE Solid-States Circuits Magazine - Summer 2020 - 50
IEEE Solid-States Circuits Magazine - Summer 2020 - 51
IEEE Solid-States Circuits Magazine - Summer 2020 - 52
IEEE Solid-States Circuits Magazine - Summer 2020 - 53
IEEE Solid-States Circuits Magazine - Summer 2020 - 54
IEEE Solid-States Circuits Magazine - Summer 2020 - 55
IEEE Solid-States Circuits Magazine - Summer 2020 - 56
IEEE Solid-States Circuits Magazine - Summer 2020 - 57
IEEE Solid-States Circuits Magazine - Summer 2020 - 58
IEEE Solid-States Circuits Magazine - Summer 2020 - 59
IEEE Solid-States Circuits Magazine - Summer 2020 - 60
IEEE Solid-States Circuits Magazine - Summer 2020 - 61
IEEE Solid-States Circuits Magazine - Summer 2020 - 62
IEEE Solid-States Circuits Magazine - Summer 2020 - 63
IEEE Solid-States Circuits Magazine - Summer 2020 - 64
IEEE Solid-States Circuits Magazine - Summer 2020 - 65
IEEE Solid-States Circuits Magazine - Summer 2020 - 66
IEEE Solid-States Circuits Magazine - Summer 2020 - 67
IEEE Solid-States Circuits Magazine - Summer 2020 - 68
IEEE Solid-States Circuits Magazine - Summer 2020 - Cover3
IEEE Solid-States Circuits Magazine - Summer 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2019
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2019
https://www.nxtbook.com/nxtbooks/ieee/mssc_2019summer
https://www.nxtbook.com/nxtbooks/ieee/mssc_2019winter
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018fall
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018summer
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018spring
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018winter
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2014
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2014
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2014
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2014
https://www.nxtbookmedia.com