IEEE Solid-State Circuits Magazine - Fall 2017 - 58

A deeper network can automatically learn
the best possible features during its training
phase, instead of relying on features handcrafted by humans.
is connected only locally to a patch
of the input tensor of size (K # K # C )
through a trainable 3-D kernel W (of
size K # K # C ) and a bias B. A formal
mathematical description to compute the outputs of a convolution
layer, l, is given as
O lfxy =

C

K

maximum of a local patch (typically
2 × 2 or 3 × 3) of output units to the
next layer. This thereby reduces the
dimension of the feature representation and creates invariance to small
shifts and distortions in the inputs.
A modern CNN consists of tens [3]
to hundreds [4] of such alternating
convolutional and max-pooling layers, typically followed by one to
three classification layers, implemented using the traditional fully
connected neurons (Figure 4).
It is important to note that the
same convolution kernel W and bias
B are used to compute all (M X M)
outputs of one slice in the output tensor. As such, every layer of the network needs only F x ^K # K # C + 1h

K

/ / / I lc^x +i h^y +j h .W lfcij + B lf .

c =0 i =0 j =0

The result of the local sum computed in this filter bank is then
passed through a nonlinearity layer,
typically a rectified linear unit (ReLU),
using the nonlinear activation function f ^u h = max ^0, u h. This output
can finally be processed by a maxpooling layer, which outputs only the

Embedded Device: Tx/Rx

Latency
Privacy
Tx Energy

Cloud: Training + Inference

Challenges for Embedded
Deep Inference

Raw Data

Scarce Resources

Classification
Result

Infinite Resources

(a)
Embedded Device: Inference

Latency
Privacy
Tx Energy
uP Energy

Cloud: Training

Training
Information

Scarce Resources

Network
Parameters

Infinite Resources

(b)
FIGURE 5: Concerns regarding user privacy, recognition latency, and energy wasted on raw
data transmission push deep learning inferences from (a) the cloud to (b) the embedded device.
Tx/Rx: transmitter/receiver; uP = microprocessor.

58

FA L L 2 0 17

IEEE SOLID-STATE CIRCUITS MAGAZINE

parameters. With K typically ranging between one and seven and F and
C on the order of tens or hundreds,
this method allows the creation of
very large networks while keeping
the number of trainable parameters
under control-all of which gave
deep learning its significant boost.
The majority of recent stateof-the-art deep learning networks
rely on such CNNs. The optimal network architecture, characterized
by the number of cascading stages
and the values of model parameters F, H, C, K, and M, varies for
each specific application. Over the
last few years, various alterations
have been proposed to this standard topology, such as, e.g., introducing feed-through connections in
ResNets [4], concatenating very small
convolutions in inception networks
[5], stacking depthwise and pointwise
convolutions in Xception networks
[6], extracting full-image dense multiscale features using DenseNets
[7], or recurrent connections in RNNs
or long short-term memories [8].
These, however, lie beyond the scope
of this tutorial.

Both the training of a deep network
and its own inferences to perform
new classifications are now typically
executed on power-hungry servers and GPUs [Figure 5(a)]. There is,
however, a strong demand to move
the inference step, in particular, out
of the cloud and into mobiles and
wearables to improve latency and
privacy issues [Figure 5(b)]. However, current devices lack the capabilities to enable deep inferences for
real-life applications.
Recent neural networks for image
or speech processing easily require
more than 100 giga-operations (GOP)/s
to 1 tera-operations (TOP)/s, as well
as the ability to fetch millions of
network parameters (kernel weights
and biases) per network evaluation.
The energy consumed in these numerous operations and data fetches is
the main bottleneck for embedded
Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Fall 2017
