IEEE Solid-States Circuits Magazine - Summer 2019 - 50

The IMC architecture is based on an array
of six-transistor static random-access memory
bit cells, and it includes periphery for
two modes of operation.
and computation control between
accelerators. Efficient integration thus
requires a proper and flexible interface design between the highly parallel IMC inputs/outputs and more
typical microprocessor blocks as well
as specialized, domain-specific architectures matched to the dataflow in
classes of applications. Although recent progress has been made in these
areas, considerable opportunities for
further research remain.

System Challenges
Computing systems must be able to
support mapping of broad sets of
applications. This raises the need
for virtualization, where the limited
hardware available is repurposed,
reconfigured, and sequenced at
runtime to efficiently support the
execution of desired computations,
typically specified through software.
This is usually done through optimizers and automatic code generators
in the compiler stack, which encapsulate algorithms for optimally mapping computations to the hardware.
Given the significantly different
physical tradeoffs presented by IMC
(see the section "IMC Fundamentals")
compared to conventional digital
architectures, the algorithms must be
carefully thought through to avoid
losing the potential gains presented
by IMC at the circuit level.

Compute
Bound

Loading
Bound

Compute
Bound

OP/s

OP/s/W

Loading
Bound

As an example, an immediate concern is the energy and latency costs
of configuring or loading stored
data in IMC. While IMC reduces the
costs of MVM computation, it doesn't
change the costs of loading data in
the memory circuits. Thus, gains are
derived only if MVM computation
costs dominate at the system level.
This depends on amortizing the dataloading costs through computational
reuse. One way to analyze this is the
widely used roof-line plot (Figure 12),
where the breakpoint between loading-bound and compute-bound
IMC operation occurs at the compute
intensity (i.e., the number of compute operations performed on each
of the loaded data), with the computation energy/throughput costs
equaling the data-loading energy/
throughput cost.
To illustrate, Figure 12 shows the
example of loading data from off-chip
DRAM. But it emphasizes the importance of evaluating IMC considering
its ba ndw idth/energ y tradeoffs
together with different applications.
Specifically, the IMC bandwidth/
latency gains push the breakpoint to
higher compute-intensity applications.
However, as an example, the trend
toward reducing operand precision
in neural-network applications [2]-[4]
can enable loading from smaller, more
efficient embedded memories or fixed

CI
1
=
OP/sComp
BWLoad
Compute Intensity

ELoad =

CI
OP/s/WComp

Compute Intensity

FIGURE 12: The roof-line plots identifying loading-bound and compute-bound regimes.

SU M M E R 2 0 19

IEEE SOLID-STATE CIRCUITS MAGAZINE

storage in IMC modules entirely, for
specific (smaller) neural networks and
use cases.

Prospects and Current
State of the Art
While IMC presents a wide range of
challenges, the initial promise it has
shown and recent approaches that
have been proposed to harness/overcome the underlying tradeoffs suggest it will be a vital area for ongoing
research, especially toward platforms
of practical scale and complexity. A
few vectors for future research and
their current states are reviewed next.

Emerging Memory Technologies
Emerging memory technologies represent a key vector for IMC research.
The primary motivation is the potential for density scaling they present
compared to SRAM. Indeed, increasing the scale of IMC based on resistive
memory technologies (RRAM, MRAM,
and PCM) has recently been demonstrated [8]-[10], [18], with even greater
progress likely as foundry options for
these technologies emerge. The primary challenge posed with regard to
the underlying SNR tradeoff in IMC is
readout of the computation result. In
particular, the technologies present
varying levels of resistance and resistance contrast, but they are generally
much more limited than the on-off
ratio or transconductance presented
by MOSFETs in SRAM bit cells. The
bit-cell computations thus possibly
lead to lower signal values, potentially
leading as well to a regime limited by
readout complexity (energy and area)
[9], which, in turn, scales in proportion
to the number of bit cells involved. In
this regime, the possible amortization of readout complexity is limited,
and IMC gains are thus strongly determined by the characteristics of the bit
cells themselves. Therefore, a critical direction for such research is the
codesign of IMC architectures with bitcell technology.
The readout complexity can have
important implications on IMC density. Specifically, crossbar architectures tend to impose more stringent

IEEE Solid-States Circuits Magazine - Summer 2019

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2019