IEEE Solid-State Circuits Magazine - Spring 2016 - 61

device. One popular option is to
never allow this, requiring the application on the host processor socket
to marshall any necessary data before
spawning an NDP task (as was done
by Pugsley et al. [28]). Another option
is to simply treat every core as being
part of a full-fledged shared-memory
multiprocessor system, i.e., every
core can issue loads and stores to any
globally visible address regardless of
whether the core resides on the host
processor or on the memory device.
This entails more software/hardware
complexity because it requires the
memory device to maintain a coherent translation look-aside buffer
(TLB) and serve as an originator of
memory requests.
This brings us to yet another key
and somewhat unsolved issue-how
is virtual memory handled? One solution, as suggested in the PEI work [2],
is to leave virtual memory management entirely up to the host processor. When a task is spawned on the
memory device, it is provided the
necessary arguments as physical
addresses; the task is not allowed to
touch data beyond the cache lines (or
pages) that were provided as arguments. Another solution, as suggested
by Pugsley et al. [28], is to organize
the data on a memory device into a
few large pages. This is a good fit
for many big data applications, and
it reduces the overheads associated
with page faults, large TLBs, etc.

What Microarchitecture Is Best
for Near Data Processing?
In most prior work, the cores on the
memory device have been designed
to be "wimpy." While some proposals incorporate full-fledged generalpurpose wimpy cores [28], such as
the 80 mW ARM Cortex A5 cores in
NDC [28] that can execute entire general-purpose map or reduce tasks,
others only implement custom functional units or accelerators [2], [3],
[15]. In the work of Ahn et al. [2],
the functional unit is only capable
of executing a single PIM-enabled
instruction, and the most complex
functional unit handles dot-product

computations for the words in a
cache line. In another example, Akin
et al. [3] design a 178-mW functional
unit that can permute data.
In addition to these fixed-function
units, we have also seen examples of
reconfigurable accelerators, such as
the use of CGRAs by Farmahini-Farahani et al. [10] and predefined accelerator primitives that can be chained
together to perform more complex
operations [15]. Also, there are proposals to combine general-purpose
wimpy cores and accelerators, e.g.,
for in-memory MapReduce workloads,
Pugsley et al. [26] execute map and
reduce phases on Arm cores, while the
sort phase between map and reduce
is handled by a fixed-function accelerator. These are all compelling design
points on the classic generality versus
efficiency spectrum.
So why is it best to pursue a
"wimpy" core instead of a low-latency
out-of-order core? An argument that
is frequently cited is the reluctance
to embed a high-power core in a
3D-stacked package for fear that it
may lead to thermal issues. But this
is often a red herring. For example,
adding a few watts to a 13-W HMC
device [18] is unlikely to pose a hazard, especially if some of the external
bandwidth can be eliminated [28]. A
more detailed study by Eckert et al.
[9] makes exactly that argument.
The more credible argument in
favor of wimpy cores is that it actually leads to higher performance as
it enables the creation of a throughput-optimized compute substrate
that can leverage the high bandwidth
afforded by NDP. As mentioned earlier, one of the main benefits of NDP
is that plugging in more memory
modules leads to more cores and
a large-scale parallel system. This
is most useful for tasks with high
degrees of parallelism. For such a
highly parallel task, the path to high
performance at a fixed power budget is to use many low-power cores,
not a few high-power cores [28]. To
be more precise, for a highly parallel
task, we can optimize throughput at
a fixed power budget by optimizing

energy per instruction [28]. Therefore, it is best to use cores or accelerators that are optimized for low
energy and not low latency. This also
enables the use of many cores or
accelerators per memory device, an
important requirement if we want to
saturate the available bandwidth.

Where Can Processors/Accelerators
Be Placed?
About two decades ago, there was
a strong push to place computation on the memory die itself. With
a potentially lower focus on cost
per bit in the future, that approach
may yet have merit. But so far, few
have chosen to revisit that direction.
A few works by Seshadri et al. [33]-
[35] have proposed small changes to
DRAM arrays to support bit manipulations and efficient data movement.
The vast majority of NDP studies in the last few years have focused
on 3D-stacked memory devices. This
approach leaves the DRAM dies relatively untouched, while leveraging
TSVs to support very high intrapackage bandwidth. By localizing the cores/
accelerators to a separate die, they can
be implemented in a superior logic
process. This approach is often touted
as the solution that offers the benefits
of NDP at relatively low cost, and that
is compatible with the natural evolution of DRAMs (3D stacking). However,
early indications are that 3D-stacked
DRAMs, especially those that include
a logic die, will not be cheap. In certain segments, the cost increase will be
well worth the higher performance.
Given the high cost of 3D-stacked
DRAM, it is worth exploring if some
(most?) of the benefits of NDP can also
be provided with conventional non3D-stacked DRAM? This is an area that
is relatively under studied and more
research needs to be done. One example proposal by Pugsley et al. [27],
NDC-Module, re-designs a DIMM by
placing many simple processor chips
on the DIMM and connecting them
to their adjacent commodity DRAM
chips. The key here is that in a conventional DIMM and server, the on-DIMM
buses can offer very high bandwidth

IEEE SOLID-STATE CIRCUITS MAGAZINE

S P R I N G 2 0 16

Table of Contents for the Digital Edition of IEEE Solid-State Circuits Magazine - Spring 2016