IEEE Solid-State Circuits Magazine - Fall 2017 - 49

using a dot product of the features
v ) (i.e.,
(xv) and a set of weights (w
w
x
)
.
/ i i i As a result, machine learn-
ing hardware research tends to focus
on reducing the cost of a multiply and
accum u late (MAC) operation.
Training involves learning these
weights from a dataset. Inference
involves performing a given task us-
ing the trained weights. In most cases,
training is done in the cloud, while infer-
ence can happen in the cloud or locally
on a device near the sensor. In latter
case, the trained weights are down-
loaded from the cloud and stored on
the device. Thus, the device needs to
be programmable in order to support
a reasonable range of tasks.

Deep Neural Networks
Rather than using handcrafted fea-
tures, the features can be directly
learned from the data, similar to
the weights in the classifier, such
that the entire system is trained end
to end. These learned features are
used in a popular form of machine
learning called called Deep Neu-
ral Networks (DNNs), also known
as deep learning [11]. DNNs deliver
higher accuracy than handcrafted
features, sometimes even better
than human-level accuracy, on a
variety of tasks by mapping inputs
to a high-dimensional represen-
tation; however, it comes at the cost
of high-computational complexity,
resulting in orders of magnitude

higher energy consumption than hand-
crafted approaches [12].
There are many forms of DNNs
(e.g., convolutional neural networks
and recurrent neural networks). For
computer vision applications, DNNs
are often composed of mu lt iple
convolutional (CONV) layers [13] as
shown in Figure 3; each layer involves
the application of multiple high-
dimensional filters to the incoming
data. With each layer, a higher-level
abstraction of the input data, called
a fe at ur e m ap, i s extracted that
preserves essential yet unique in -
formation. Modern DNNs are able
to achieve superior performance
by employing a very deep hierar-
chy of layers on the order of tens
to hundreds.

The output of the final CONV layer
is typically processed by fully con-
nected (FC) layers for classification.
In FC layers, the filter and input fea-
ture map are the same size so that
there is a unique weight for each input
feature value. In between CON V and
FC layers, additional functions can
be added, such as pooling and nor-
malization [14]. In addition, a nonlin-
ear function, such as a rectified linear
unit (ReLU) [15], is applied after each
CONV and FC layer. Overall, convo-
lutions account for over 90% of the
run time and energy consumption in
modern DNNs for computer vision.
Table 1 compares modern DNNs,
with a popular neural net from the
1990s, LeNet-5 [16]. Today's DNNs
use more layers (i.e., deeper) and are

Modern DNNs: 5-1,000 Layers
Low-Level
Features

CONV
Layer

Convolution

CONV
Layer

Nonlinearity

1-3 Layers

High-Level
Features

FC
Layer

Normalization

Classes

Pooling

FIGURE 3: DNNs are composed of several CONV layers followed by FC layers.

TABLE 1. A SUMMARY OF POPULAR DNNs [16], [18]-[21]. ACCURACY IS MEASURED BASED ON THE TOP-FIVE ERRORS ON
IMAGENET [22].
METRICS

LeNet 5

AlexNet

VGG-16

GoogLeNet (V1)

ResNet-50

Accuracy

n/a

16.4

7.4

6.7

5.3

CONV layers

Weights

2.6 thousand

2.3 million

14.7 million

6 million

23.5 million

MACs

283 thousand

6.66 billion

15.3 billion

1.43 billion

3.86 billion

FC layers

Weights

58 thousand

58.6 million

124 million

1 million

2 million

MACs

58 thousand

58.6 million

124 million

1 million

2 million

Total weights

60 thousand

61 million

138 million

7 million

25.5 million

Total MACs

341 thousand

724 million

15.5 billion

1.43 billion

3.9 billion

IEEE SOLID-STATE CIRCUITS MAGAZINE

FA L L 2 0 17

Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Fall 2017