IEEE Solid-State Circuits Magazine - Spring 2016 - 58

and latency-optimized general-purpose cores are fairly well understood
[16]. There will be a steady trickle of
architecture innovations for generalpurpose processors, but these are
unlikely to disrupt the relatively flat
average improvement curve for large
benchmark suites. Without significant annual improvements, computer
systems end up as commodities sold
at low margins.
What then drives the computing industry forward? What is the

Accelerators are being actively
studied in architecture research circles. Accelerators have already been
designed for popular data-intensive
algorithms, e.g., data partitioning
[43], database queries [44], sort [26],
and machine learning [6].
As computational throughput on a
processor increases, with help from
accelerators, there is a corresponding
demand for higher memory capacity and bandwidth. Enterprise-class
workloads, e.g., SAP HANA [30] and

For a few decades now, researchers have
considered off-loading parts of an application
to a processor embedded in the memory system.

and compression. Such features are
compatible with most memory technologies, while some features (e.g.,
those dealing with wearout) are an
especially good fit for emerging nonvolatile memory (NVM) technologies.
While minimal amounts of logic may
be placed within memory dies, the
more significant features will likely be
placed in separate logic chips. These
logic chips can be coupled with memory dies either with through silicon
vias (TSVs) in a 3D-stacked package or
with on-board traces in a dual in-line
memory module (DIMM) form factor.
This article discusses the features that
can be meaningfully added to memory
devices and the impact they can have
on server architectures.

Memory System Features
motivation for hardware/architecture innovation?
A shift toward specialization is
inevitable. There will likely be a significant low-margin market for general-purpose commodity systems
and a second significant high-margin market for specialized systems.
This is how the automobile industry has operated for decades. To
some extent, this is already a reality today in the computing market.
A desktop computer can be built for
around US$500; this is how we build
a cluster to do many architecture
simulations in parallel. But a single
graphics processing unit (GPU) card
can cost ten times that amount,
and this is what we use to run our
machine learning algorithms.
Two phenomena will serve as the
drivers of the computing industry
in the coming decade. The first is
the growing focus on accelerators.
The second is a shift toward feature-rich memory systems. Both of
these paths are relatively less traveled, i.e., they have the potential to
uncover large benefits. Combined,
these two phenomena will form the
basis for specialized systems that
can significantly outperform previous-generation systems and command a higher price tag.

S P R I N G 2 0 16

SAS in-memory analytics [31], are well
known for demanding low-latency
access for massive data sets. This is
an increasingly prevalent phenomenon as several industries grapple with
analytics that can convert big data
into big money.
In this era of big data processing,
a large fraction of overall time and
energy is expended in data access and
data movement. Following Amdahl's
law, the memory/storage system is
clearly where system innovations
can have the largest impact. This is
especially true because the memory
system has not been a target of architecture innovations for the past three
decades. We are long overdue for specialized memory systems that are not
constrained by standards or by an
unwavering focus on cost per bit.
Memory system innovations can
help a vendor distinguish its products from the competition. The new
currency for a memory product will
therefore be features. Cost per bit is a
fine metric for the commodity generalpurpose space, but it will be a secondary metric for specialized systems.
So what features can one place
within the memory system? These
features may include, for example,
simple processing units, accelerators, logic for reliability, security,

IEEE SOLID-STATE CIRCUITS MAGAZINE

Figure 1 summarizes the overall
approach of a feature-rich memory
system. I will classify memory system features into two main groups:
processing features and auxiliary
features. The first group provides
logic to execute parts of the application, and this logic can take the form
of a general-purpose processor or
an application-specific accelerator.
The second group provides logic to
perform auxiliary operations that
are independent of the application
but critical for overall system efficiency. Such operations may include
wear leveling, encryption, compression, and coding.

Processing Features
For a few decades now, researchers
have considered off-loading parts of
an application to a processor embedded in the memory system. The area
of processing-in-memory (PIM) was
heavily researched in the 1990s but
remained dormant for a decade after
that for a variety of reasons, most
notably, the economics of integrating logic and DRAM on a single die.
The area has now reemerged [4],
thanks to improvements in technology
[e.g., three-dimensional (3D) stacking],
the demands of emerging workloads
(e.g., big data workloads that benefit
from high memory bandwidth), and, as

Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Spring 2016