IEEE Solid-States Circuits Magazine - Fall 2022 - 24

through a common upper level memory
[34] or a network on chip (NoC) with
a variety of possible topologies: ring,
torus, and so on [1], [28]. The different
cores could reside on the same chip
but could also be distributed across
multiple chiplets, which are then combined
on interposer [1], [7], [28].
These different parallelization
schemes all come with their own
benefits and downsides. The more
explicit the split is between the different
cores, the more expensive it
is to share data between these cores,
yet the more freedom the scheduler
has to increase utilization. It is hence
important in the design of such
multi-core architectures to perform a
careful tradeoff between scheduling
efficiency and hardware efficiency.
Indeed, also, the hardware implementation
itself has many degrees
of freedom, such as the number of
cores; the size of the cores' MAC arrays;
their memory hierarchy; and
the spatial and temporal data reuse
supported within each core. These
design parameters can, moreover, be
identical for all cores (homogeneous
multi-core) or vary from core to core
(heterogeneous multi-core).
Outlook: Will It Be Homogenous
or Heterogeneous?
We are just at the verge of this multicore
era of ML processors. The research
field is still experimenting
with a wide variety of multi-core architectures.
One central question in
this quest is whether the field will
evolve to homogeneous or heterogeneous
multi-core architectures.
Homogeneous multi-core architectures
are most widespread and
benefit from reduced design complexity
and scheduling/compilation
complexity. Especially when exploiting
chiplets integrated on interposers,
homogeneous architectures
drastically bring down mask and
production costs. Yet it is difficult to
develop a single processor core that
performs well across all workloads.
For example, data sharing patterns in
depthwise layers, pointwise layers,
and fully connected layers can vary
widely. By keeping the core sizes
relatively small, utilization across all
these workloads can be kept at a decent
level. Yet this is, of course, at the
expense of less spatial data reuse,
and hence, more memory accesses
and resulting energy cost.
An emerging class of heterogeneous
multi-core architectures foregoes
the desire to develop a single
accelerator core that fits all needs. Instead,
by combining a variety of cores,
they aim to support each workload
Computing
Array
Local Mem.
Computing
Array
Local Mem.
Computing
Array
Local Mem.
Shared L1 Memory
Shared L2 Memory
System Bus
Shared Control
(a)
Core
Local Ctr.
Local Mem.
L1 Mem.
L2 Mem.
Core
Local Ctr.
Local Mem.
L1 Mem.
L2 Mem.
Core
Local Ctr.
Local Mem.
L1 Mem.
L2 Mem.
Core
Local Ctr.
Local Mem.
L1 Mem.
L2 Mem.
System Bus
Main Memory
(b)
External Mem.
Interposer
(c)
FIGURE 5: Multi-core ML accelerators differ in terms of the granularity at which the additional degrees of parallelization are added:
from parallel SIMD compute arrays; over parallel complete accelerator cores; to parallel accelerator chiplets on integrated on interposers.
(a) Parallel SIMD computing arrays. (b) Parallel accelerator cores. (c) Parallel chiplets on interposer. Mem.: memory; Ctr. control.
24
FALL 2022
IEEE SOLID-STATE CIRCUITS MAGAZINE
External Mem.
Computing
Array
Local Mem.

IEEE Solid-States Circuits Magazine - Fall 2022

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Fall 2022