IEEE Circuits and Systems Magazine - Q3 2021 - 32

Lastly we present several future research directions for CIM such
as incremental
learning, monolithic 3D integration,
rable design and potential security vulnerabilities.
1. Overview of Compute-in-Memory
I
n recent years, deep learning has achieved remarkable
success in a variety of applications such as image classification,
object detection, speech recognition, and natural
language processing, etc. [1]. The success on software
algorithms has triggered a wave of designing specialized
hardware accelerators for efficient implementations of
deep neural network (DNN) models from cloud to edge. At
cloud, the mainstream platform for deep learning acceleration
today is primarily graphic-processing-unit (GPU) due
to its computational accuracy and programming flexibility.
A cluster of GPUs is a common setting for DNN training,
which typically consumes hundreds to thousands
of Watts power, making it viable to data-center enviro
nment. To improve energy efficiency, application-specific-integrated-circuit
(ASIC) or domain-specific architecture
solutions (e.g. Google's tensor-processing-unit,
TPU [2]) are becoming more popular, which could be
tailored for both cloud and edge applications. The grand
challenge for deep learning acceleration is frequent data
movement back and forth between compute units and
memory units [3], similar as the memory wall problem in
conventional von-Neumann architecture. GPU and TPU
(or the equivalence) are not solving the memory wall
problem as compute cores or processing elements (PEs)
are still separated from off-chip main memory or on-chip
global buffer. The operation which takes the most part of
DNN processing is vector-matrix multiplication (VMM)
between the input vector and weight matrix, which essentially
performs multiply-and-accumulate (MAC) operation.
To this end, in-memory computing or compute-inmemory
(CIM) is proposed as a promising paradigm since
it emerges computation directly into memory sub-arrays
[4]. As we show later in this article, CIM is well positioned
to accelerate VMM for deep learning inference. For background
materials regarding deep learning algorithm and
architecture co-optimization, readers are referred to this
article [5]. For basics of emerging non-volatile memory
(eNVM) devices and circuits, readers are referred to this
article [6]. For material/device-level advances of CIM technologies,
readers are referred to this article [7]. For CIM's
broader applications beyond deep learning such as spiking
neural network or scientific computing, readers are
referred to this article [8].
In this review, firstly we will survey representative
CIM macros that are implemented into silicon, and
reconfiguthen
will discuss general design challenges and lessons
learned from these prototype chips. Then we will use an
in-house benchmark simulator to provide projections to
the full system-level performance, as many of today's CIM
demonstrations are at macro-level with single sub-array
or a limited number of sub-arrays. Finally we will discuss
several emerging topics for future research directions.
1.1. Principle of CIM
VMM's efficiency, if performed in the CIM manner, could
be remarkably boosted by the crossbar nature of memory
sub-array with perpendicular input rows and output
columns, as shown in Fig. 1. The weights of a DNN model
could be mapped as the conductance of memory cells in
sub-array, while the input vector is loaded in parallel as
voltage to the rows, then multiplication is done in analog
fashion (i.e. input voltage multiplied by weight conductance),
and current summation along columns is used to
generate the output vector. Here we use a black-box to
conceptually represent the memory cell that could have
multilevel states (for multi-bit weight), but actual implementation
is often done with a series transistor to form
a 1-transistor-1-resistor (1T1R) structure. Sometimes a
pass-gate transistor is directly used as the weight element
if its channel conductance is determined by a latch
state as in the static random access memory (SRAM).
Analog-to-digital converter (ADC) at edge of the subarray
is commonly employed to convert the weighted
sum (typically referred to as partial sum due to the limited
sub-array size) to binary bits for subsequent digital
processing (e.g. shift-and-add, accumulation, activation,
and pooling). Therefore, CIM is mixed-signal compute
in nature with analog compute core and digital peripheral
processing. In principle, VMM could be done in
fully parallel fashion if asserting all the rows and all the
columns simultaneously. In practice, multiple rows/columns
could be partially turned on due to limited sensing
resolution of ADC or mismatch between the column
pitch and the ADC size.
Multi-bit weight/input/output precision could be
supported in CIM. Depending on the memory cell's
precision, multi-bit weight may be split into multiple
cells. For example, an 8-bit weight could be represented
by 4 memory cells (in 4 columns) if using
2-bit per cell. Then the outputs after ADC will need
to go through shift-and-add process to reconstruct
the significance across multiple columns. The input
precision could be encoded as analog voltage levels
or as multiple cycles that are sequentially loaded to
the rows. Due to limited dynamic range of the input
Shimeng Yu, Hongwu Jiang, Shanshi Huang, Xiaochen Peng, and Anni Lu are with the School of Electrical and Computer Engineering, Georgia Institute
of Technology. (Corresponding author email: shimeng.yu@ece.gatech.edu).
32
IEEE CIRCUITS AND SYSTEMS MAGAZINE
THIRD QUARTER 2021

IEEE Circuits and Systems Magazine - Q3 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2021