IEEE Circuits and Systems Magazine - Q4 2022 - 32

during inference. Activations in ImageNet neural networks,
for example, can typically be quantized to 8 bits
after training without losing significant inference accuracy
[31]. Can analog systems exploit this to perform
accurate ImageNet inference with no more than 8 bits of
ADC resolution? The answer is yes, and the key enabler
is dot product proportionality. While a dot product (or
activation) may tolerate quantization to 8 bits, this property
might be lost if the same information is encoded in a
quantity that is not proportional to the original dot product.
Ensuring proportionality between analog outputs
and dot products connects the ADC resolution requirement
to the algorithm's intrinsic precision requirements.
Dot-product proportionality largely follows from weight
proportionality, with the additional requirement that
the current subtraction in differential cells be conducted
in analog. This will be explored in Section 6.
3.3. Full Precision Fallacy
The FPG requires the ADC to have a unique level for every
possible output of an analog MVM, and thus match the
precision of a digital processor. To be compatible with
practical ADC resolutions (~8 bits), the FPG bounds the
amount of computation that can be executed in the analog
domain before digitization. This is expressed by Equation
(2). There are two fundamental problems with the FPG.
First, the FPG is only meaningful if the accumulated
cell error on all bit lines is below the LSB of the ADC.
When cell errors are present, the analog input to the
ADC does not necessarily have Bout bits of precision as
given by Equation (2). Thus, in practice, satisfying the
FPG requires not only the correct ADC resolution but
also sufficiently accurate memory cells to ensure that
the ADC resolution is fully utilized. Some early work on
in situ MVM explicitly set the ADC resolution to match
the expected level of accumulated cell error, using fewer
bits than required by the FPG [21], [52].
Second, the FPG is imposed at the level of the analog
MVM kernel, typically without full consideration of its
utility for the accuracy of neural network inference. By
focusing on the precision of an individual kernel rather
than end-to-end system requirements, the FPG creates
inefficiencies, as predicted by the end-to-end argument
of Saltzer et al. [53]. Specifically, in Section 6, we show
that in systems with dot product proportionality, the FPG
is too conservative. In these systems, the ADC resolution
requirement can be decoupled from the hardware configuration
and dictated instead by the end-to-end accuracy
of the neural network application. Fortuitously, the resolution
requirement of ImageNet neural networks is also
~8 bits [31]. Removing the constraints of the FPG enables
much more analog computation to be done for the same
ADC resolution, improving energy efficiency.
32
IEEE cIrcuIts and systEms magazInE
4. Accuracy Evaluation Method
This section describes the methodology for inference
accuracy evaluation for the results presented in the
remaining sections. Unless otherwise stated, neural networks
are quantized to 8-bit precision, a common use
case for inference [31], [35].
4.1. Mapping Weights to Conductances
We assume a simple, parameter-less procedure to map
a layer's weights onto device conductances. First, the
floating-point weight matrix WFP is scaled by the maximum
absolute value maxWFP into the range [−1, +1].
To quantize the weights to 8 bits, these weights are further
scaled to the range [−127, +127], then rounded to
integers in this range. The same process is used for the
weights of all layers.
The quantized weight matrix W, with integer values
in the range [−127, +127], is then decomposed into
one or more non-negative integer-valued matrices that
can be mapped to conductances. The specific decomposition
depends on the method used to handle negative
weights (offset subtraction vs. differential cells)
and encode weight precision (with or without bit slicing).
Several examples are shown in Fig. 5 for an 8-bit
matrix. Although these representations are functionally
equivalent in the absence of analog errors, they differ
greatly in their sensitivity to these errors. These methods
represent the majority of proposed analog accelerators.
4.1.1. Mapping Without Bit Slicing
Fig. 5(a) shows how an 8-bit matrix W is mapped using offset
subtraction without bit slicing, following the equation:
WW
 
xx x2
prog
27I
(4)
where I is the identity matrix and Wprog is a strictly nonnegative
8-bit matrix in the range [1, 255] that can be
mapped directly onto conductances. This matrix has
an offset such that a zero weight in W is mapped to a
value of 27 in Wprog. This offset is subtracted from the
MVM result to represent negative weights. Computing
the offset term requires summing the elements of

x,
which can be done digitally. Shafiee et al. [56] also proposed
an analog computation of this offset with a " unit
column " , which will be evaluated in Section 5.2. We note
that a value of −128 in W can be mapped by a value of 0
in Wprog, but this state is left unused.
Fig. 5(b) shows how the same matrix is mapped using
differential cells, following the equation:
WW W
 
xx x2
(5)
where the strictly positive weight matrices W+
W- have 7-bit values in the range [0,127], and are
and
fourth quartEr 2022
IEEE Circuits and Systems Magazine - Q4 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q4 2022
