IEEE Circuits and Systems Magazine - Q3 2021 - 35

Among eNVMs, RRAM is widely used in many prototype chips due to its
relatively larger on/off ratio than STT-MRAM, less power consumption
than PCM, and more foundry availability than FeFET.
memory (RRAM) [28], phase change memory (PCM) [29],
spin-transfer-torque magnetic random access memory
(STT-MRAM) [30], spin-orbit-torque magnetic random access
memory (SOT-MRAM) [31], ferroelectric field effect
transistor (FeFET) [32] and electrochemical random access
memory (ECRAM) [33]. In the past decade, industry
has heavily invested in research and development
of eNVMs. Nowadays commercial fabrication processes
of eNVMs are available, e.g. TSMC's 40 nm RRAM [34],
28 nm RRAM [35], and 22 nm RRAM [36], Intel's 22 nm
RRAM [37], TSMC's 40 nm PCM [38], STMicroelectronics'
28 nm PCM [39], TSMC's 22 nm STT-MRAM [40],
Intel's 22 nm STT-MRAM [41], Globalfoundries' 22 nm
STT-MRAM [42], Samsung's 28 nm STT-MRAM [43], Globalfoundries'
FeFET at 28 nm [44] and 22 nm [45]. To our
best knowledge, most of these industry-mature eNVM
processes offer 1-bit per cell. Multi-bit per cell capability
is still under the exploration, while some research progresses
were made towards 5-bit per cell RRAM [46] or
even 10-bit per cell PCM [47].
1.4. Inference Versus Training
So far, most of the reported CIM designs are targeted
for inference only where the DNN model is pre-trained
by software. One-time programming is required to load
the weights into memory sub-arrays. We believe that
such inference engine is the first tangible target for
CIM based accelerators. Improvements in throughput
and energy efficiency are highly desirable for real-time
decision making and power-constrained edge devices.
Potential drawbacks of CIM such as degraded accuracy
may be tolerable for certain types of edge applications.
Capability of processing information locally at the frontend
is beneficial for saving bandwidth and energy to
send data (possibly wirelessly) back to cloud. Security
and privacy is another motivation to develop smart
edge platforms as users may be reluctant to share personal
data. We will primarily focus on the CIM designs
for edge inference in this article.
On the contrary, training means that weights are
learned on-the-fly during runtime. Here we would point
out subtle differences between two concepts: in-situ
training and on-chip training. It is well known that training
generally requires higher weight precision than inference
only [48]. In-situ training refers to the case where a
single memory cell itself has sufficiently high precision
to represent a weight, and weight update occurs as the
THIRD QUARTER 2021
switching between intermediate states of the memory
cell. The weight increment is calculated by the peripheral
circuitry and is translated as the number of write
pulses. To simplify the peripheral circuitry design, identical
pulsing scheme is preferred over varying-amplitude/width
pulsing scheme. In this case, the nonlinear
and asymmetric weight update behavior (device conductance
vs. number of identical write pulses) of the reported
eNVMs significantly hampers in-situ training accuracy
for algorithms based on the stochastic gradient descent
method [49] [50]. A linear and symmetric weight update
behavior is thus highly demanding for eNVMs [51].
On-chip training does not necessarily require a single
memory cell to represent a high precision weight.
Instead, it could employ multiple cells to form from the
most significant bit (MSB) to the least significant bit
(LSB). The weight increment is calculated by the peripheral
circuitry and converted to binary bits to be written
from MSB to LSB. In this case, on-chip training is feasible,
similarly as writing the data into digital memory array.
Though the density benefits of eNVMs for such on-chip
training will diminish to some extent, but it remains an
option for SRAM based design [52].
2. Recent Progresses in Chip-Level Demonstrations
Generally, CIM could be implemented with SRAM or
eNVMs. Among eNVMs, RRAM is widely used in many
prototype chips due to its relatively larger on/off ratio
than STT-MRAM, less power consumption than PCM,
and more foundry availability than FeFET. In this article,
we will only survey RRAM based CIM macros. Before
we survey the prototype chips, we would clarify some
ambiguities in the literature regarding the metrics to
evaluate hardware accelerators. Terra-operations per
second per watt (TOPS/W) is often used as the standard
metric for energy efficiency. Because most of the ASIC
designs (including digital MAC, near-memory compute
and CIM) support fixed-precision computation (but with
different precisions for different designs), it is better to
normalize TOPS/W to the same input/weight precision
for a fair comparison. In this article, we will use 1-bit by
1-bit MAC (defined as 2 OP) for normalization. Therefore,
8-bit by 8-bit MAC is naturally 64 times higher TOPS/W
than 1-bit by 1-bit MAC. We will also use the same definition
of OP for throughput instead of using frames per
second (FPS), as FPS is dataset dependent. As a reference
(assuming normalized 1-bit by 1-bit MAC), NVidia's
IEEE CIRCUITS AND SYSTEMS MAGAZINE
35
IEEE Circuits and Systems Magazine - Q3 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2021
