IEEE Solid-State Circuits Magazine - Fall 2017 - 51

use temporal architectures such as
SIMD or SIMT to perform the MACs
in parallel. All the arithmetic logic
units (ALUs) share the same control
and memory (register file). On these
platforms, all classifications are
represented by a matrix multiplica-
tion. The CONV layer in a DNN can
also be mapped to a matrix multi-
plication using the Toeplitz matrix.
Software libraries that optimize for
matrix multiplications can be used
to accelerate processing on CPUs
(e.g., OpenBLAS and Intel MKL) and
GPUs (e.g., cuBL A S and cuDNN).
The matr i x multiplications c a n
be further sped up by applying
transforms such as fast Fourier trans-
form [25], [26] and Winograd [27]
to the data to reduce the number
of multiplications.

Specialized Hardware
Specialized hardware provides an
opportunity to optimize the data
movement (i.e., data flow) to mini-
mize accesses from the expensive
levels of the memory hierarchy and
maximize data reuse at the low-cost
levels of the memory hierarchy. Fig-
ure 6 shows the memory hierarchy of
the spatial architecture in Figure  5,
where each ALU processing element
(PE) has a local memory (register file)
on the order of several kilobytes and
a shared memory (global buffer) on
the order of several hundred k ilo -
bytes. The global buffer communi-
cates with the off-chip memory (e.g.,
DRAM). Data movement is allowed
between the PEs using an on-chip net-
work to reduce accesses to the global
buffer and the off-chip memory.
The data flows of all three types
of data (feature map, filter weights,
and partial sums) affect energy con-
sumption. Various data flows have
been demonstrated in recent works
[28]-[39], which differ in terms of
the type of data that moves and the
type of data that remains station-
ary in the register file of the PE [40].
The row stationary data flow, which
considers the energy consumption
of all three data types, reduces the
energy consumption by 1.4× to 2.5×

The key metrics for embedded machine learning
are accuracy, energy consumption, throughput,
and cost.

compared to the other data flows for
the CONV layers [41].

Opportunities in Joint Algorithm
and Hardware Design
The machine learning algorithms
c a n b e m o d if ie d to m a ke t h e m
more hardware friendly by reducing
com putation, data movement, and
storage requirements, while main-
taining accuracy.

Reduce Precision
GPUs and CPUs commonly use a 32-b
floating point as the default repre-
sentation. For inference, it is possible
to use fixed point with reduced bit
width for energy and area savings,
and increased throughput, without
affecting accuracy.
For instance, for object detec-
tion using handcrafted HOG features,
only 11 bits are required per fea-
ture vector and only 5 bits per SVM
weight [42]. For DNN inference, recent
commercial hardware uses 8-b inte-
ger operations [43]. Custom hard-
ware can be used to exploit the fact
that the minimum bit widths varies
per layer for energy savings [44]
or increased throughput [45]. With

Global
Buffer

DRAM

more significant changes to the net-
work, it is possible to reduce the bit
width of DNNs to 1-b at the cost of
reduced accuracy [46], [47].

Sparsity
Increasing sparsity in the data reduces
storage and computation cost. For
SVM classification, the weights can
be projected onto a basis such that
the resulting weights are sparse for a
2× reduction in number of multiplica-
tions [42]. For feature extraction, the
input image can be made sparse by
preprocessing for a 24% reduction in
power consumption [48].
For DNNs, the number of MACs and
weights can be reduced by removing
weights through a process called prun-
ing. This was first explored in [49]
where weights with minimal impact
on the output were removed. In [50],
pruning is applied to modern DNNs
by removing small weights. However,
removing weights does not necessar-
ily lead to lower energy. Accordingly,
in [51], weights are removed based
on an energy model [52] to directly
minimize energy consumption.
Specialized hardware in [42] and
[53]-[55] exploits sparse weights for

PE

PE

PE

ALU

Fetch Data to Run
a MAC Here

Normalized Energy Cost

0.5-1.0 kB
NoC: 200-1,000 PEs

RF

PE

ALU

1× (Reference)

ALU

1×

ALU

100-500 kB Buffer

ALU

DRAM

ALU

2×
6×
200×

FIGURE 6: Memory hierarchy and data movement energy for a spatial architecture [41].

IEEE SOLID-STATE CIRCUITS MAGAZINE

FA L L 2 0 17

51



Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Fall 2017

IEEE Solid-State Circuits Magazine - Fall 2017 - Cover1
IEEE Solid-State Circuits Magazine - Fall 2017 - Cover2
IEEE Solid-State Circuits Magazine - Fall 2017 - 1
IEEE Solid-State Circuits Magazine - Fall 2017 - 2
IEEE Solid-State Circuits Magazine - Fall 2017 - 3
IEEE Solid-State Circuits Magazine - Fall 2017 - 4
IEEE Solid-State Circuits Magazine - Fall 2017 - 5
IEEE Solid-State Circuits Magazine - Fall 2017 - 6
IEEE Solid-State Circuits Magazine - Fall 2017 - 7
IEEE Solid-State Circuits Magazine - Fall 2017 - 8
IEEE Solid-State Circuits Magazine - Fall 2017 - 9
IEEE Solid-State Circuits Magazine - Fall 2017 - 10
IEEE Solid-State Circuits Magazine - Fall 2017 - 11
IEEE Solid-State Circuits Magazine - Fall 2017 - 12
IEEE Solid-State Circuits Magazine - Fall 2017 - 13
IEEE Solid-State Circuits Magazine - Fall 2017 - 14
IEEE Solid-State Circuits Magazine - Fall 2017 - 15
IEEE Solid-State Circuits Magazine - Fall 2017 - 16
IEEE Solid-State Circuits Magazine - Fall 2017 - 17
IEEE Solid-State Circuits Magazine - Fall 2017 - 18
IEEE Solid-State Circuits Magazine - Fall 2017 - 19
IEEE Solid-State Circuits Magazine - Fall 2017 - 20
IEEE Solid-State Circuits Magazine - Fall 2017 - 21
IEEE Solid-State Circuits Magazine - Fall 2017 - 22
IEEE Solid-State Circuits Magazine - Fall 2017 - 23
IEEE Solid-State Circuits Magazine - Fall 2017 - 24
IEEE Solid-State Circuits Magazine - Fall 2017 - 25
IEEE Solid-State Circuits Magazine - Fall 2017 - 26
IEEE Solid-State Circuits Magazine - Fall 2017 - 27
IEEE Solid-State Circuits Magazine - Fall 2017 - 28
IEEE Solid-State Circuits Magazine - Fall 2017 - 29
IEEE Solid-State Circuits Magazine - Fall 2017 - 30
IEEE Solid-State Circuits Magazine - Fall 2017 - 31
IEEE Solid-State Circuits Magazine - Fall 2017 - 32
IEEE Solid-State Circuits Magazine - Fall 2017 - 33
IEEE Solid-State Circuits Magazine - Fall 2017 - 34
IEEE Solid-State Circuits Magazine - Fall 2017 - 35
IEEE Solid-State Circuits Magazine - Fall 2017 - 36
IEEE Solid-State Circuits Magazine - Fall 2017 - 37
IEEE Solid-State Circuits Magazine - Fall 2017 - 38
IEEE Solid-State Circuits Magazine - Fall 2017 - 39
IEEE Solid-State Circuits Magazine - Fall 2017 - 40
IEEE Solid-State Circuits Magazine - Fall 2017 - 41
IEEE Solid-State Circuits Magazine - Fall 2017 - 42
IEEE Solid-State Circuits Magazine - Fall 2017 - 43
IEEE Solid-State Circuits Magazine - Fall 2017 - 44
IEEE Solid-State Circuits Magazine - Fall 2017 - 45
IEEE Solid-State Circuits Magazine - Fall 2017 - 46
IEEE Solid-State Circuits Magazine - Fall 2017 - 47
IEEE Solid-State Circuits Magazine - Fall 2017 - 48
IEEE Solid-State Circuits Magazine - Fall 2017 - 49
IEEE Solid-State Circuits Magazine - Fall 2017 - 50
IEEE Solid-State Circuits Magazine - Fall 2017 - 51
IEEE Solid-State Circuits Magazine - Fall 2017 - 52
IEEE Solid-State Circuits Magazine - Fall 2017 - 53
IEEE Solid-State Circuits Magazine - Fall 2017 - 54
IEEE Solid-State Circuits Magazine - Fall 2017 - 55
IEEE Solid-State Circuits Magazine - Fall 2017 - 56
IEEE Solid-State Circuits Magazine - Fall 2017 - 57
IEEE Solid-State Circuits Magazine - Fall 2017 - 58
IEEE Solid-State Circuits Magazine - Fall 2017 - 59
IEEE Solid-State Circuits Magazine - Fall 2017 - 60
IEEE Solid-State Circuits Magazine - Fall 2017 - 61
IEEE Solid-State Circuits Magazine - Fall 2017 - 62
IEEE Solid-State Circuits Magazine - Fall 2017 - 63
IEEE Solid-State Circuits Magazine - Fall 2017 - 64
IEEE Solid-State Circuits Magazine - Fall 2017 - 65
IEEE Solid-State Circuits Magazine - Fall 2017 - 66
IEEE Solid-State Circuits Magazine - Fall 2017 - 67
IEEE Solid-State Circuits Magazine - Fall 2017 - 68
IEEE Solid-State Circuits Magazine - Fall 2017 - 69
IEEE Solid-State Circuits Magazine - Fall 2017 - 70
IEEE Solid-State Circuits Magazine - Fall 2017 - 71
IEEE Solid-State Circuits Magazine - Fall 2017 - 72
IEEE Solid-State Circuits Magazine - Fall 2017 - 73
IEEE Solid-State Circuits Magazine - Fall 2017 - 74
IEEE Solid-State Circuits Magazine - Fall 2017 - 75
IEEE Solid-State Circuits Magazine - Fall 2017 - 76
IEEE Solid-State Circuits Magazine - Fall 2017 - 77
IEEE Solid-State Circuits Magazine - Fall 2017 - 78
IEEE Solid-State Circuits Magazine - Fall 2017 - 79
IEEE Solid-State Circuits Magazine - Fall 2017 - 80
IEEE Solid-State Circuits Magazine - Fall 2017 - 81
IEEE Solid-State Circuits Magazine - Fall 2017 - 82
IEEE Solid-State Circuits Magazine - Fall 2017 - 83
IEEE Solid-State Circuits Magazine - Fall 2017 - 84
IEEE Solid-State Circuits Magazine - Fall 2017 - 85
IEEE Solid-State Circuits Magazine - Fall 2017 - 86
IEEE Solid-State Circuits Magazine - Fall 2017 - 87
IEEE Solid-State Circuits Magazine - Fall 2017 - 88
IEEE Solid-State Circuits Magazine - Fall 2017 - 89
IEEE Solid-State Circuits Magazine - Fall 2017 - 90
IEEE Solid-State Circuits Magazine - Fall 2017 - 91
IEEE Solid-State Circuits Magazine - Fall 2017 - 92
IEEE Solid-State Circuits Magazine - Fall 2017 - 93
IEEE Solid-State Circuits Magazine - Fall 2017 - 94
IEEE Solid-State Circuits Magazine - Fall 2017 - 95
IEEE Solid-State Circuits Magazine - Fall 2017 - 96
IEEE Solid-State Circuits Magazine - Fall 2017 - 97
IEEE Solid-State Circuits Magazine - Fall 2017 - 98
IEEE Solid-State Circuits Magazine - Fall 2017 - 99
IEEE Solid-State Circuits Magazine - Fall 2017 - 100
IEEE Solid-State Circuits Magazine - Fall 2017 - 101
IEEE Solid-State Circuits Magazine - Fall 2017 - 102
IEEE Solid-State Circuits Magazine - Fall 2017 - 103
IEEE Solid-State Circuits Magazine - Fall 2017 - 104
IEEE Solid-State Circuits Magazine - Fall 2017 - 105
IEEE Solid-State Circuits Magazine - Fall 2017 - 106
IEEE Solid-State Circuits Magazine - Fall 2017 - 107
IEEE Solid-State Circuits Magazine - Fall 2017 - 108
IEEE Solid-State Circuits Magazine - Fall 2017 - 109
IEEE Solid-State Circuits Magazine - Fall 2017 - 110
IEEE Solid-State Circuits Magazine - Fall 2017 - 111
IEEE Solid-State Circuits Magazine - Fall 2017 - 112
IEEE Solid-State Circuits Magazine - Fall 2017 - 113
IEEE Solid-State Circuits Magazine - Fall 2017 - 114
IEEE Solid-State Circuits Magazine - Fall 2017 - 115
IEEE Solid-State Circuits Magazine - Fall 2017 - 116
IEEE Solid-State Circuits Magazine - Fall 2017 - 117
IEEE Solid-State Circuits Magazine - Fall 2017 - 118
IEEE Solid-State Circuits Magazine - Fall 2017 - 119
IEEE Solid-State Circuits Magazine - Fall 2017 - 120
IEEE Solid-State Circuits Magazine - Fall 2017 - 121
IEEE Solid-State Circuits Magazine - Fall 2017 - 122
IEEE Solid-State Circuits Magazine - Fall 2017 - 123
IEEE Solid-State Circuits Magazine - Fall 2017 - 124
IEEE Solid-State Circuits Magazine - Fall 2017 - 125
IEEE Solid-State Circuits Magazine - Fall 2017 - 126
IEEE Solid-State Circuits Magazine - Fall 2017 - 127
IEEE Solid-State Circuits Magazine - Fall 2017 - 128
IEEE Solid-State Circuits Magazine - Fall 2017 - 129
IEEE Solid-State Circuits Magazine - Fall 2017 - 130
IEEE Solid-State Circuits Magazine - Fall 2017 - 131
IEEE Solid-State Circuits Magazine - Fall 2017 - 132
IEEE Solid-State Circuits Magazine - Fall 2017 - 133
IEEE Solid-State Circuits Magazine - Fall 2017 - 134
IEEE Solid-State Circuits Magazine - Fall 2017 - 135
IEEE Solid-State Circuits Magazine - Fall 2017 - 136
IEEE Solid-State Circuits Magazine - Fall 2017 - 137
IEEE Solid-State Circuits Magazine - Fall 2017 - 138
IEEE Solid-State Circuits Magazine - Fall 2017 - 139
IEEE Solid-State Circuits Magazine - Fall 2017 - 140
IEEE Solid-State Circuits Magazine - Fall 2017 - 141
IEEE Solid-State Circuits Magazine - Fall 2017 - 142
IEEE Solid-State Circuits Magazine - Fall 2017 - 143
IEEE Solid-State Circuits Magazine - Fall 2017 - 144
IEEE Solid-State Circuits Magazine - Fall 2017 - 145
IEEE Solid-State Circuits Magazine - Fall 2017 - 146
IEEE Solid-State Circuits Magazine - Fall 2017 - 147
IEEE Solid-State Circuits Magazine - Fall 2017 - 148
IEEE Solid-State Circuits Magazine - Fall 2017 - Cover3
IEEE Solid-State Circuits Magazine - Fall 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2023
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2022
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2021
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_spring2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_winter2020
https://www.nxtbook.com/nxtbooks/ieee/mssc_fall2019
https://www.nxtbook.com/nxtbooks/ieee/mssc_summer2019
https://www.nxtbook.com/nxtbooks/ieee/mssc_2019summer
https://www.nxtbook.com/nxtbooks/ieee/mssc_2019winter
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018fall
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018summer
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018spring
https://www.nxtbook.com/nxtbooks/ieee/mssc_2018winter
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2017
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2016
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2015
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_winter2014
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_fall2014
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_summer2014
https://www.nxtbook.com/nxtbooks/ieee/solidstatecircuits_spring2014
https://www.nxtbookmedia.com