IEEE Solid-State Circuits Magazine - Fall 2017 - 50

several orders of magnitude larger
in terms of compute and storage. A
more detailed discussion on DNNs
can be found in [17].

Impact of Difficulty of Task
on Complexity
The difficulty of the task must be
considered when comparing differ-
ent hardware platforms for machine
learning as the size of the classifier or
network (i.e., number of weights) and
the number of MACs tend to be larger
for more difficult tasks and thus
require more energy. For instance,
the task of classifying handwritten
digits from the MNIST dataset [23]
is much simpler than classifying an
object into one of a 1,000 classes

MNIST

in the ImageNet data set [22] (Fig-
ure 4). Accordingly, LeNet-5, which
is designed for digit classification,
requires much less storage and com-
pute than the larger DNNs in Table 1,
which are designed for the 1,000-
class image classification task. Thus,
hardware platforms should only be
compared when performing machine
learning tasks of similar difficulty
and accuracy, ideally, the same task
with the same accuracy.

Challenges
The key metrics for embedded machine
learning are accuracy, energy con-
sumption, throughput, and cost. The
challenge is to address all these re-
quirements concurrently.

ImageNet

FIGURE 4: The MNIST (ten classes, 60,000 training, and 10,000 testing) [23] versus ImageNet (1,000 classes, 1.3 million training, and 100,000 testing) [22] dataset.

Temporal Architecture
(SIMD/SIMT)

Spatial Architecture
(Data Flow Processing)

Memory Hierarchy

ALU

Control

ALU

As previously discussed, the ac-
curacy of the machine le a r n i ng
algorithm should be measured for
a well-defined task on a sufficiently
large dataset (e.g., ImageNet). Energy
consumption is often dominated by
data movement as memory access
consumes significantly more energy
than computation [24]. This is par-
ticularly challenging for machine
learning as the high-dimensional
representation and filters increase
the amount of data generated, and
the programmability needed to sup-
port different applications, tasks,
and networks means that the weights
also need to be read and stored. In
this article, we will discuss various
methods that reduce data movement
to minimize energy consumption.
The throughput is dictated by the
amount of computation, which also
increases with the dimensionality of
the data. In this article, we will dis-
cuss various transforms that can be
applied to the data to reduce the num-
ber of required operations.
The cost is dictated by the amount
of storage required on the chip. In
this article, we will discuss various
methods to reduce storage costs such
that the area of the chip is reduced,
while maintaining low off-chip mem-
ory bandwidth.
Currently, state-of-the-art DNNs
consume orders of magnitude higher
energy than other forms of embedded
processing (e.g., video compression)
[12]. We must exploit opportunities
at multiple levels of hardware design
to address all these challenges and
close this energy gap.

Opportunities in Architectures
ALU

ALU

The MAC operations in both the fea-
ture extraction (CONV layers in a
DNN) and classification (for both
DNN and handcrafted features) can
be easily parallelized. Two com-
mon highly parallel compute para-
digms that can be used are shown
in Figure 5.

CPU and GPU Platforms
FIGURE 5: Highly parallel compute paradigms with multiple ALUs.

FA L L 2 0 17

IEEE SOLID-STATE CIRCUITS MAGAZINE

Central processing units (CPUs) and
graphics processing units (GPUs)

Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Fall 2017