IEEE Circuits and Systems Magazine - Q3 2023 - 51

chiplet size choice. If a chiplet's x- and y-dimension are
each scaled up by a factor of S, the compute capacity
scales up by a factor of approximately S2, but the chiplet
I/O shoreline only scales up by a factor of S, allowing
the I/O bandwidth to scale up by approximately S. This
back-of-envelope calculation suggests that chiplet size
may have to be kept smaller or the disparity between
compute and I/O can become even larger.
A mapping strategy was developed for Nvidia's MCP
to divide the weights into parts and allocate them to different
chiplets [54]: 1) allocate output channels (different
kernels) across columns of chiplets and 2) divide the
input channels into parts and allocate them across rows
of chiplets. To carry out the computation, the channels
of the inputs are divided into parts and distributed to
appropriate rows of chiplets. This mapping strategy provides
data reuse and reduction: 1) weights are cached
and reused within a chiplet; 2) input activations are
reused between multiple kernels within a chiplet; and
3) output psums are reduced in the channel dimension
before going out of the chiplet. Such a mapping strategy
is essential for reducing the I/O usage and removing the
I/O bottleneck in the DNN MCP.
B. Heterogeneous Integration
While homogeneous tiling of DNN chiplets solves the
problem of scaling up DNN hardware to support larger
DNN models, it does not address the problem of scaling
out DNNs, i.e., extending DNNs to novel uses, e.g., DNNs
used as a building block to support new applications.
Besides scaling out DNNs, new operators can be added
to DNNs in the future to enhance its capability, making it
difficult to design a truly future-proof DNN chiplet.
We argue the importance of factoring computation
into types, e.g., common operations and special operations,
in considering chiplet-based system partitioning.
As examples, CONV and FC layers are common and
compute-heavy operations; and batch normalization
and activation functions are special operations and
relatively lightweight compared to CONV and FC layers.
The control loops and data organization outside
of NN processing to support different tasks are also
special operations. This factoring exercise naturally
leads to heterogeneous chiplets, e.g., an accelerator
chiplet that supports common and compute-heavy
operations, and a processor or FPGA chiplet that can
be programmed to support special operations. Using
this approach, accelerator chiplets can be made to target
common kernels that are unlikely to change over
time, allowing us to extend the useful lifetime of these
chiplets. Processor and FPGA chiplets can be used to
complement the accelerator chiplets to complete system
implementations.
An example of heterogeneous integration is the MCP
consisting of an FPGA with the PETRA systolic array
chiplet [55] as illustrated in Fig. 23. The PETRA chiplet
measures 3 mm2 in an Intel 22 nm technology. It integrates
tiles of systolic arrays to provide up to 1,024
MACs/cycle (FP16) or 1.43 TFLOPS (FP16) [55].
The PETRA MCP is built on Intel's embedded multidie
interconnect bridge (EMIB) [56], [57], a silicon bridge
that connects an FPGA chiplet and an external chiplet.
The silicon bridge provides a high routing density, enabling
the use of parallel links of moderate speed. The
I/O design for moderate-speed links can be made much
simpler than high-speed serial I/Os, and it can even be
made entirely digital [58]. A digital link is more reliable
and can be ported to different technologies with ease. In
the MCP design, a digital advanced interface bus (AIB)
link [57], [58] was adopted with full swing, supporting a
short-reach of 3 mm at 2 Gbps/pin. Thanks to the short
reach and simple design, an AIB I/O consumes less than
1 pJ/b [58]. An AIB channel assembles 40 pins for an
aggregate bandwidth of 80 Gbps. Using a dense bump
pitch of 55 μm, an AIB channel occupies approximately
300 μm of die edge. The PETRA chiplet utilizes 8 AIB
Figure 23. Illustration of the concept of integrating an FPGA with the PETRA chiplet. Figure reused from [55] ©2021 IEEE.
THIRD QUARTER 2023
IEEE CIRCUITS AND SYSTEMS MAGAZINE
51

IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023