IEEE Solid-States Circuits Magazine - Fall 2022 - 20

1,000
10,000
100,000
100
10
H. Jia ISSCC (2021)
0.1
1
0.01
0.00001 0.0001
A. Garofalo CoRR (2022)
CHIMERA (2022)
Eyeriss (2016)
ENVISION (2017)
0.001
DianNao (2014)
0.01
0.1 110
Power (W)
100
1,000
10,000
FIGURE 1: Recent accelerator evolutions in the quest for higher throughput at the edge increasingly exploit multi-core implementations [1],
[4], [5], [6], [7], [16], [19], [24], [28], [32], [33]. Heter.: heterogeneous; Homo.: homogeneous; TOPS: tera operations per second;
TOPS/W: tera operations per second/watt.
how a specific workload is scheduled
(or " mapped " ) on the processor.
It is therefore important to understand
the difference between peak
efficiency and workload efficiency.
The peak efficiency of a processor
is characterized when all processing
elements are actively participating
in the computation, under a specific
Layer
Temporal Data Reuse
//Off-Chip Storage
for OX = [0, 4):
for OY = [0, 4):
//On-Chip SRAM
for K = [0, 16):
for C = [0, 16):
for Fx = [0, 3):
for Fy = [0, 3):
//On-Chip RF
parfor Cu = [0, 16):
parfor Cu = [0, 16):
Inputs/Outputs
Update
Temporally
Weight
Stationary
Reuse
Temporally
C0
Input (OX+Fx, OY+Fy, 16∗C+Cu)∗
Output (OX, OY, K)+=
Cu Unroll
Spatially
Weight (Fx, Fy, 16∗C+Cu, 16∗K+Ku)
Spatial Data Reuse
C0
C1
C14
C15
C1
MACMAC
MACMAC
MACMAC
MACMAC
C14 C15
MACMAC
MACMAC
MAC Array
MACMAC
MACMAC
One Input Shared Among MAC Row
FIGURE 2: The single-core template architecture, demonstrating opportunities for temporal
and spatial reuse when mapping a single CONV layer.
20
FALL 2022
IEEE SOLID-STATE CIRCUITS MAGAZINE
On-Chip SRAM/RF
(One or More Memory Levels)
Weights
Outputs
Inputs
workload that is perfectly matched
to the processor's architecture: ensuring
that all input and output dimensions
are exact multiples of the
processor's datapath; all input and
output tensors fit inside the on-chip
memories; and so forth. Actual ML
workloads, however, vary widely in
layer topologies and dimensions. As
Off-Chip Storage
a result, it is not possible to build a
single processor architecture that
perfectly fits all possible workloads,
and hence, a processor will typically
not be able to reach its full peak performance
on actual workloads. This
gap can be quantified by assessing
the underutilization of the processor
under realistic workloads.
First, a processor can be spatially
underutilized, when the spatial data
reuse pattern of the datapath mismatches
with the NN's layer dimensions
that can be unrolled along the
datapath's spatial dimensions, as
visualized in Figure 3(a) (left). Specifically,
the spatial utilization is
computed as the fraction of compute
elements that are performing relevant
computations for the specific
workload under study whenever the
datapath is not idle.
Second, a processor can also be
temporally underutilized for a given
workload, when the datapath has to
stall because the processor has to
load or store data before the next
compute cycle can start [Figure 3(b)].
Specifically, the temporal utilization
is computed as the fraction of clock
cycles the datapath is computing
Tesla NPU (2020)
Single Core
Single-Core to Multi-Core:
Increased Performance
Increased Scheduling Complexity
Multi-Core (Homo.)
Multi-Core (Heter.) Multiple Chiplets
Simba (2021)
Diana (2022)
C. Lin ISSCC (2020)
A. Agrawal ISSCC (2021)
100 TOPS/W
10 TOPS/W
1 TOPS/W
100 GOPS/W
Performance (TOPS)
Partial Output Accumulated Across MAC Column
IEEE Solid-States Circuits Magazine - Fall 2022

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Fall 2022
