IEEE Solid-State Circuits Magazine - Fall 2017 - 62

A straightforward way to benefit from the network's fault tolerance is to perform the computations
at reduced computational accuracy
with limited recognition loss. Typical benchmarks can be run at a 1-9-b
fixed point rather than a 32-b floating
point at lower than 1% accuracy loss
[18]. This is possible by quantizing
all weights of a floating-point-trained
network before execution. Improved
results can be obtained when introducing quantization during the training step itself [19], [38], resulting in
smaller or lower-precision networks
for the same application accuracy. As
an extreme example, networks have
been specifically trained to operate with only 1-b representations of
weights alone [20] as well as with
both weights and activations [20],
[21] wherein all multiplications can
be replaced by efficient XNOR operations [22]. In [20], a binary-weight
version of ImageNet is only 2.9% less
accurate (in top-1 accuracy) than the
full-precision AlexNet [3].
This observation can lead to major
energy savings, as current CPU and
GPU architectures operate using
32-16-b floating-point number formats. Reducing precision from 32-b
floating point to low precision not
only reduces computational energy
but also minimizes the storage and
data-fetching cost needed for network

weights and intermediate results.
Moreover, for very low bit widths,
this even allows the replacement of
multipliers that have several data
values with a common weight factor
via preloaded lookup tables [10]. As
a result, all custom CNN accelerators
operate in fixed point. While most
processors operate at constant 16-,
12-, or 8-b word lengths, some recent
implementations support variable
word-length computations, wherein
the processor can change the used
computational precision from operation to operation [23], [10], [24]. This
accommodates for the observation
that the optimal word length for a
deep network strongly varies from
application to application and is
even shown to differ across various
layers of a single deep network [18]
[Figure 10(a)].
Energy-efficient variable-resolution processors have been realized
using a technique termed dynamic
voltage-accuracy-frequency scaling
[25] to jointly reduce the switching
activity, supply voltage, and parallelization scheme when computational resolution drops [Figure 10(c)].
This results in a scaling of the system's energy consumption, which is
super-linear with the computational
resolution [Figure 10(b)], thus allowing every network layer to run at its
own minimal energy point. Reduced

bit-width implementations all exploit
the deep network's tolerance to faults
in a deterministic way.
Another school of thought targets
energy savings through tolerating
nondeterministic statistical errors.
This can be accomplished by executing the convolutional kernels in the
noisy analog domain [26]. Alternatively, in the digital domain, stochastic fault tolerance can be exploited
by operating the circuits [27] and/
or memory [28] in the energy-efficient near-threshold regions. In this
region, circuit delays as well as
memory failures suffer from large
variation. Yet the networks can tolerate such stochastic behavior up to
a certain limit. Such circuits are combined with circuit monitors that constantly assess and control the circuit's
fault rate [28].
Finally, the operational circumstances can strongly influence the network's tolerance to approximations. In
a given classification application, the
quality of the inputs might change
dynamically, or some classes might be
easier to observe than others. If one
tries to train one common network
that performs acceptably under all
possible circumstances and classes,
a large, complex, energ y-hungr y
network topology would be needed.
Recent work, however, promotes the
training of hierarchical or staged

AlexNet on ImageNet

y3
0

Relative Power

Quantization (Bits)

6
4

Uniform at 100%

2
0

Nonuniform at 99%
2

4
6
Layer Number
(a)

33× Gain
at 1% RMSE

16 Bit
10-2

10-4
10-6

6 Bit

y 2 y 1/0 y 0/0

x 0/0
x 1/0

p 0/0

p 1/0

p 2/0

1 Bit
10-4
10-2
Computational Precision
(b)

p 3/0
100

(c)

FIGURE 10: (a) When quantizing all weight and data values in a floating point AlexNet uniformly, the network can run at 9-b precision.
Lower precision can be achieved without significant classification accuracy loss by running every layer at its own optimal precision. This allows
(b) saving power in the function of computational precision and (c) building multipliers whose energy consumption scales drastically with computational precision, through reduced activity factor and critical path length.

FA L L 2 0 17

IEEE SOLID-STATE CIRCUITS MAGAZINE

Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Fall 2017