IEEE Circuits and Systems Magazine - Q2 2021 - 24

increase). Floating-point capabilities will also be supported
in the DSP58 tiles of the next-generation Xilinx Versal
architecture [80].
The second direction targets increasing the density of
low-precision integer multiplication specifically for DL inference
workloads. Prior work has demonstrated the use
of low-precision fixed-point arithmetic (8-bit and below)
instead of fp32 at negligible or no accuracy degradation,
but greatly reduced hardware cost [81]-[83]. However,
the required precision is model-dependent and can even
vary between different layers of the same model. As a result,
FPGAs have emerged as an attractive solution for
DL inference due to their ability to implement custom
precision datapaths, their energy efficiency compared
to GPUs, and their lower development cost compared
to custom ASICs. This has led both academic researchers
and FPGA vendors to investigate adding native support
for low-precision multiplication to DSP blocks. The
authors of [84] enhance the fracturability of an Intel-like
DSP block to support more int9 and int4 multiply and
MAC operations, while keeping the same DSP block routing
interface and ensuring its backward compatibility.
The proposed DSP block could implement four int9 and
eight int4 multiply/MAC operations along with Arria10-like
DSP block functionality at the cost of 12% DSP
block area increase, which is equivalent to only 0.6% increase
in total die area. This DSP block increased the performance
of 8-bit and 4-bit DL accelerators by
13 . # and
16 . # while reducing the utilized FPGA resources by 15%
and 30% respectively, compared to an FPGA with DSPs
that do not natively support these modes of operation.
Another academic work [85] enhanced a Xilinx-like DSP
block by including a fracturable multiplier array instead
of the fixed-precision multiplier in the DSP48E2 block to
support int9, int4 and int2 precisions. It also added
a FIFO register file and special dedicated interconnect
between DSP blocks to enable more efficient standard,
point-wise and depth-wise convolution layers. Shortly
after, Intel announced that the same int9 mode of operation
will be added to the next-generation Agilex DSP
block along with half-precision floating-point (fp16) and
brain float (bfloat16) precisions [86]. Also, the nextgeneration
Xilinx Versal architecture will natively support
int8 multiplications in its DSP58 tiles [80].
Throughout the years, the DSP block architecture has
evolved to best suite the requirements of key application
domains of FPGAs, and provide higher flexibility such
that many different applications can benefit from its capabilities.
The common focus across all the steps of this
evolution was reusing multiplier arrays and routing ports
as much as possible to best utilize both these costly resources.
However, this becomes harder with the recent
divergence in the DSP block requirements of key FPGA
24
IEEE CIRCUITS AND SYSTEMS MAGAZINE
application domains between high-precision floatingpoint
in HPC, medium-precision fixed-point in communications,
and low-precision fixed-point in DL. As a result,
Intel has recently announced an AI-optimized FPGA, the
Stratix 10 NX, which replaces conventional DSP blocks
with AI tensor blocks [87]. The new tensor blocks drop
the support for legacy DSP modes and precisions that
were targeting the communications domain and adopt
new ones targeting the DL domain specifically. This tensor
block significantly increases the number of int8 and
int4 MACs to 30 and 60 per block respectively, at almost
the same die size [88]. Feeding all multipliers with inputs
without adding more routing ports is a key concern. Accordingly,
the NX tensor block introduces a double-buffered
data reuse register network that can be sequentially
loaded from a smaller number of routing ports, while allowing
common DL compute patterns to make the best
use of all available multipliers [89]. The next-generation
Speedster7t FPGA from Achronix will also include a machine
learning processing (MLP) block [90]. It supports a
variety of precisions from int16 down to int3 in addition
to fp24, fp16 and bfloat16 floating-point formats.
The MLP block in Speedster7t will also feature a tightly
coupled BRAM and circular register file that enable the
reuse of both input values and output results. Each of
these tightly integrated memory banks has a 72-bit external
input but can be configured to have an up-to 144-bit
output that feeds the MLP's multiplier arrays, reducing
the number of required routing ports by
2 .#
F. System-Level Interconnect: Network-on-Chip
FPGAs have continuously increased both in capacity and
in the bandwidth of their external IO interfaces such as
DDR, PCIe and Ethernet. Distributing the data traffic between
these high-speed interfaces and the ever-larger
soft fabric is a challenge. This system-level interconnect
has traditionally been built by configuring parts of the
FPGA logic and routing to implement soft buses that realize
multiplexing, arbitration, pipelining and wiring between
the relevant endpoints. These external interfaces
operate at higher frequencies than the FPGA fabric can
achieve, and therefore the only way to match their bandwidth
is to use wider (soft) buses. For example, a single
channel of high-bandwidth memory (HBM) has a 128-bit
double data rate interface operating at 1 GHz, so a
bandwidth-matched soft bus running at 250 MHz must
be 1024 bits wide. With recent FPGAs incorporating up to
8 HBM channels [91] as well as numerous PCIe, Ethernet
and other interfaces, system level interconnect can rapidly
use a major fraction of the FPGA logic and routing
resources. In addition, system-level interconnect tends to
span large distances. The combination of very wide and
physically long buses makes timing closure challenging
SECOND QUARTER 2021
IEEE Circuits and Systems Magazine - Q2 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2021
