IEEE Circuits and Systems Magazine - Q2 2021 - 86

storage, and maximizes the utilization of relatively slow
external memory. AI Engine processors have several
communication mechanisms to support different application
patterns, including shared memory and streaming
communication.
Shared memory communication allows direct communication
between AI Engines, with processors reading
and writing values in adjacent memories (represented
by red arrows in Figure 7). This access is facilitated
by the physical layout of AI Engine processors and memories
in a checkerboard pattern, shown in Figure 9. To
reduce synchronization costs, each data memory is also
associated with a set of dedicated hardware locks that
can be used to synchronize this communication with
neighbours. Typically, communication happens through
a pair of ping-pong buffers, where one processor is writing
to one of the buffers while another processor is reading
from another. When a processor is done accessing
one buffer, it releases the lock associated with that buffer
and attempts to acquire the lock associated with the
other buffer. In this way, both processors can proceed in
a pipelined fashion, with each processor operating concurrently
on different blocks of data.
The stream-switched interconnect enables communication
between non-neighboring processors, supported
by the local DMA engines in each processor tile.
These DMA engines access processor data memory and
are capable of sending or receiving data over the AXI
stream switched interconnect, represented by blue arrows
of Figure 7. The DMA engines also synchronize
with AI Engine processors through the hardware locks.
DMA-based communication can be overlapped with
processor computation, requiring 2 independent sets of
ping-pong buffers.The stream-switched interconnect is
also able to broadcast data from one source to multiple
destinations simultaneously.
A final communication mechanism is a direct cascade
stream connection between horizontal neighbours,
represented by grey arrows in Figure 7, enabling
values stored in the wide vector accumulation registers
to be communicated over a dedicated stream connection
without loss of precision.
Communication between the AI Engine processors
and the rest of the device can be performed in several
ways. One mechanism allows stream switch connections
directly into the programmable logic, enabling a
wide, high bandwidth connection with programmable
logic IP. It is also possible to leverage the hardened
Network-on-Chip (NoC) [102], which connects the major
building blocks of the Versal device (processor subsystem,
programmable logic, the AIE-array) to each other
and to the external memory and other I/O interfaces as
shown in Figure 10. The Versal NoC is packet switched
and supports both memory mapped and streaming
connections with a deterministic routing flow. The
NoC provides a persistent addressable interconnect
that unifies all resources of the platform, enabling
more of the programmable logic to be used for custom
logic. The AI Engine stream switch interconnect can
pass streaming data directly into the Noc, or hardened
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Mem
AI Engine
Processor
Figure 9. The tile layout of the AI Engine processor array. Each AI Engine processor is adjacent to 4 memory blocks: one in its
own tile and 3 in adjacent tiles.
86
IEEE CIRCUITS AND SYSTEMS MAGAZINE
SECOND QUARTER 2021

IEEE Circuits and Systems Magazine - Q2 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2021