IEEE Circuits and Systems Magazine - Q2 2021 - 85

shifted, rounded or saturated before storing it in the local
memory. 32-bit single-precision vector floating-point
operations are also supported.
For 16-bit and 8-bit datatypes, the AI Engine vector
unit implements two-dimensional SIMD operation
(Figure 8) organized into rows and columns of multiplications.
Each element of the output vector is computed
from a row of multiplications which are added
together, forming a single vector lane. The result of
this reduction is accumulated in the wide vector accumulator
register. The number of columns of a vector
operation is dependent on the precision of the
operand. 8-bit vector operations execute as 16 rows
with 8 columns, yielding 128 8-bit MACs per clock.
16 -bit vector operat ions are
configured either as 8 lanes with
4 columns or 16 lanes with 2 columns,
yielding 32 16-bit MAC per
clock. Various combinations of
complex data types are also natively
supported [16].
For area-efficient implementation,
the AI Engine architecture
requires all vector memory operations
to be on aligned addresses.
To enable register-level data reuse
and operations on unaligned
data, the vector datapath includes
a fused programmable permute
network. The permute network enables
flexible selection of vector
data between the vector register
file and the SIMD datapath.
16 KB
Program
Memory
C. Communication and Memory Efficiency
In addition to arithmetic, the AI Engine architecture
also has specialized communication and local storage
capabilities, illustrated by arrows and dark grey
blocks of Figure 7. These capabilities are designed to
allow concurrent pipelined communication and compute,
with each vector datapath executing efficiently
from local memory, while the next set of data to be processed
is being loaded into memory. Unlike most multiprocessor
architectures that rely on a cache hierarchy
to reduce external memory bandwidth, the AI Engine
architecture puts explicit data movement in the hands
of programmers and tools. This approach avoids performance
impacts from stalls, reduces redundant data
32 b Scalar
RISC Unit
HW
Locks
Floating-Point
Vector Unit
Fixed-Point
Vector Unit
32 KB
Data
Memory
DMA
Unit
Figure 7. The AI Engine processor tile [16] combines a VLIW processor with fixedpoint
and floating-point vector instructions, shown in green, with a local compiler-managed
scratchpad memory, shown in grey, and an stream-oriented network-on-chip,
shown in yellow.
Local Memory (32 kB)
Vector Register File (256 B)
Permute Network
() +
× () ++× () =
×
() +
× () ++× () =
×
acc Lane 0
acc Lane 1
Shift
Round
Saturate
() +
× () ++× () =
×
acc Lane 15
Figure 8. The AI Engine datapath enables 2 D SIMD through accumulation rows and multiply add rows combined with a permute
(shuffle) network enabling vector register-level data-reuse.
SECOND QUARTER 2021
IEEE CIRCUITS AND SYSTEMS MAGAZINE
85
Load-Store Unit

IEEE Circuits and Systems Magazine - Q2 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2021