IEEE Circuits and Systems Magazine - Q3 2021 - 52

SRAM at the latest technology node (e.g. 5 nm or beyond) still offers the
best compute efficiency for high-performance applications.
scratch does not make sense for eNVMs because of the
stringent requirements on the asymmetry/ nonlinearity
and the relatively expensive cost for write operations.
Incremental learning on edge devices is possible that
assumes that the model is pre-trained and just needs
the fine tuning at the edge to adapt to the local environment
or user preference. Partial training (e.g. only
the fully connected layers) could relax the burdens on
eNVMs. For example, a mixed design [128] with
eNVMs for convolutional layers and SRAM for fully
connected layers is feasible. The weights in eNVMs do
not need update, while the model fine-tuning only occurs
in the SRAM.
SRAM at the latest technology node (e.g. 5 nm or beyond)
still offers the best compute efficiency for highperformance
applications. Today's eNVMs such as
RRAM can be tuned to multilevel (possibly by iterative
programming), and offer advantages over SRAM (e.g.
low leakage and non-volatility) for standby-frequent applications.
If equipped with always-on circuitry for activation
signal detection [129], eNVM based accelerators
could be turned on instantly for the subsequent information
processing (e.g. feature extraction for sensory
signals). From the manufacturing cost point of view,
eNVMs at 22 nm/28 nm node may be a sweet spot for
many IoT applications.
The slight inference accuracy degradation in CIM
architectures is caused by process variation and especially
the ADC offset. This may bring overhead of testing
phase for retraining from chip to chip. Lightweight and
fast retraining technique remains a research topic to be
further explored. eNVMs such as RRAM based inference
engine still faces other design challenges such as high
write voltage and low on-state resistance, ADC overhead,
intermediate
state stability, etc. Further device
engineering is necessary, e.g. to explore other attractive
device candidates such as FeFET.
The benefits of CIM architectures are limited by
the available embedded memory capacity. Once the
chip area is not enough to fit the entire model on chip,
the benefits diminish if weights need to be reloaded
[95]. Today's SRAM or eNVMs are suited for moderate
sized problem with DNN model parameters in the
range of MB to tens of MB. To hold more weights onchip,
one possible future research direction is to explore
monolithic 3D integration (M3D). M3D designs
could partition the ADCs and other peripheral logic
modules (e.g. accumulator/buffers) on the silicon sub52
IEEE
CIRCUITS AND SYSTEMS MAGAZINE
strate, and place one or multiple eNVM tiers (with its
neighboring periphery, such as MUX/decoder) at the
top back-end-of-line (BEOL) layers [130]. One benefit
of such M3D integration is that plenty of room is
now open at the silicon substrate where more ADCs
could be placed underneath the eNVM tier. The ADC
bottleneck could be relieved because none or less
column sharing is required thereby improving parallelism
and throughput. In addition, area-consuming
advanced offset cancellation techniques could be applied
to ADCs to mitigate the inference accuracy degradation
as aforementioned. M3D fabrication is becoming
wafer-scale ready with recent breakthroughs
in the silicon recrystallization by laser annealing
technique [131]. BEOL-compatible oxide transistors
[132] also offer an option to integrate the memory tier
on top of the logic tier.
Another research topic is to enable reconfigurability
in CIM architectures. The representative CIM architectures
such as ISACC [133], PRIME [134], PipeLayer
[135] and TIME [136] can generally adapt to different
DNN models during the design time. It should be clarified
that though these reported architectures were
typically evaluated using various DNN models, in fact
each model was implemented on a separate custom
chip with different on-chip resources but following the
same principle of architecture. As algorithm development
cycle is shorter than hardware design cycle, it is
preferred to have runtime reconfigurability in CIM architectures
[137]. Network-on-chip or equivalent routing
schemes may be needed to enable different dataflow
for different networks. Reconfigurable architecture
coupled with M3D design may offer the ultimate solutions
to CIM accelerators.
Lastly, there are potential software/hardware security
vulnerabilities with CIM accelerators, as the
weights of the DNN model are stored in memory cells.
The DNN model that is stored in CIM inference chip is
identified as the valuable asset to be protected, as it
requires extensive resources (huge amount of dataset
and labels, clusters with GPUs), time (days to weeks)
and power (thousands of Watts) to train a large-scale
DNN model. If the adversary could easily read-out or
probe the weights from the inference chip, or simply
run the input-output gathering experiments to reverse
engineer the DNN model, he could resell it in the software
format or clone it into another counterfeit chip.
Countermeasures to these emerging attack models are
THIRD QUARTER 2021
IEEE Circuits and Systems Magazine - Q3 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2021
