IEEE Circuits and Systems Magazine - Q2 2021 - 85
shifted, rounded or saturated before storing it in the local
memory. 32-bit single-precision vector floating-point
operations are also supported.
For 16-bit and 8-bit datatypes, the AI Engine vector
unit implements two-dimensional SIMD operation
(Figure 8) organized into rows and columns of multiplications.
Each element of the output vector is computed
from a row of multiplications which are added
together, forming a single vector lane. The result of
this reduction is accumulated in the wide vector accumulator
register. The number of columns of a vector
operation is dependent on the precision of the
operand. 8-bit vector operations execute as 16 rows
with 8 columns, yielding 128 8-bit MACs per clock.
16 -bit vector operat ions are
configured either as 8 lanes with
4 columns or 16 lanes with 2 columns,
yielding 32 16-bit MAC per
clock. Various combinations of
complex data types are also natively
supported [16].
For area-efficient implementation,
the AI Engine architecture
requires all vector memory operations
to be on aligned addresses.
To enable register-level data reuse
and operations on unaligned
data, the vector datapath includes
a fused programmable permute
network. The permute network enables
flexible selection of vector
data between the vector register
file and the SIMD datapath.
16 KB
Program
Memory
C. Communication and Memory Efficiency
In addition to arithmetic, the AI Engine architecture
also has specialized communication and local storage
capabilities, illustrated by arrows and dark grey
blocks of Figure 7. These capabilities are designed to
allow concurrent pipelined communication and compute,
with each vector datapath executing efficiently
from local memory, while the next set of data to be processed
is being loaded into memory. Unlike most multiprocessor
architectures that rely on a cache hierarchy
to reduce external memory bandwidth, the AI Engine
architecture puts explicit data movement in the hands
of programmers and tools. This approach avoids performance
impacts from stalls, reduces redundant data
32 b Scalar
RISC Unit
HW
Locks
Floating-Point
Vector Unit
Fixed-Point
Vector Unit
32 KB
Data
Memory
DMA
Unit
Figure 7. The AI Engine processor tile [16] combines a VLIW processor with fixedpoint
and floating-point vector instructions, shown in green, with a local compiler-managed
scratchpad memory, shown in grey, and an stream-oriented network-on-chip,
shown in yellow.
Local Memory (32 kB)
Vector Register File (256 B)
Permute Network
() +
× () ++× () =
×
() +
× () ++× () =
×
acc Lane 0
acc Lane 1
Shift
Round
Saturate
() +
× () ++× () =
×
acc Lane 15
Figure 8. The AI Engine datapath enables 2 D SIMD through accumulation rows and multiply add rows combined with a permute
(shuffle) network enabling vector register-level data-reuse.
SECOND QUARTER 2021
IEEE CIRCUITS AND SYSTEMS MAGAZINE
85
Load-Store Unit
IEEE Circuits and Systems Magazine - Q2 2021
Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2021
Contents
IEEE Circuits and Systems Magazine - Q2 2021 - Cover1
IEEE Circuits and Systems Magazine - Q2 2021 - Cover2
IEEE Circuits and Systems Magazine - Q2 2021 - Contents
IEEE Circuits and Systems Magazine - Q2 2021 - 2
IEEE Circuits and Systems Magazine - Q2 2021 - 3
IEEE Circuits and Systems Magazine - Q2 2021 - 4
IEEE Circuits and Systems Magazine - Q2 2021 - 5
IEEE Circuits and Systems Magazine - Q2 2021 - 6
IEEE Circuits and Systems Magazine - Q2 2021 - 7
IEEE Circuits and Systems Magazine - Q2 2021 - 8
IEEE Circuits and Systems Magazine - Q2 2021 - 9
IEEE Circuits and Systems Magazine - Q2 2021 - 10
IEEE Circuits and Systems Magazine - Q2 2021 - 11
IEEE Circuits and Systems Magazine - Q2 2021 - 12
IEEE Circuits and Systems Magazine - Q2 2021 - 13
IEEE Circuits and Systems Magazine - Q2 2021 - 14
IEEE Circuits and Systems Magazine - Q2 2021 - 15
IEEE Circuits and Systems Magazine - Q2 2021 - 16
IEEE Circuits and Systems Magazine - Q2 2021 - 17
IEEE Circuits and Systems Magazine - Q2 2021 - 18
IEEE Circuits and Systems Magazine - Q2 2021 - 19
IEEE Circuits and Systems Magazine - Q2 2021 - 20
IEEE Circuits and Systems Magazine - Q2 2021 - 21
IEEE Circuits and Systems Magazine - Q2 2021 - 22
IEEE Circuits and Systems Magazine - Q2 2021 - 23
IEEE Circuits and Systems Magazine - Q2 2021 - 24
IEEE Circuits and Systems Magazine - Q2 2021 - 25
IEEE Circuits and Systems Magazine - Q2 2021 - 26
IEEE Circuits and Systems Magazine - Q2 2021 - 27
IEEE Circuits and Systems Magazine - Q2 2021 - 28
IEEE Circuits and Systems Magazine - Q2 2021 - 29
IEEE Circuits and Systems Magazine - Q2 2021 - 30
IEEE Circuits and Systems Magazine - Q2 2021 - 31
IEEE Circuits and Systems Magazine - Q2 2021 - 32
IEEE Circuits and Systems Magazine - Q2 2021 - 33
IEEE Circuits and Systems Magazine - Q2 2021 - 34
IEEE Circuits and Systems Magazine - Q2 2021 - 35
IEEE Circuits and Systems Magazine - Q2 2021 - 36
IEEE Circuits and Systems Magazine - Q2 2021 - 37
IEEE Circuits and Systems Magazine - Q2 2021 - 38
IEEE Circuits and Systems Magazine - Q2 2021 - 39
IEEE Circuits and Systems Magazine - Q2 2021 - 40
IEEE Circuits and Systems Magazine - Q2 2021 - 41
IEEE Circuits and Systems Magazine - Q2 2021 - 42
IEEE Circuits and Systems Magazine - Q2 2021 - 43
IEEE Circuits and Systems Magazine - Q2 2021 - 44
IEEE Circuits and Systems Magazine - Q2 2021 - 45
IEEE Circuits and Systems Magazine - Q2 2021 - 46
IEEE Circuits and Systems Magazine - Q2 2021 - 47
IEEE Circuits and Systems Magazine - Q2 2021 - 48
IEEE Circuits and Systems Magazine - Q2 2021 - 49
IEEE Circuits and Systems Magazine - Q2 2021 - 50
IEEE Circuits and Systems Magazine - Q2 2021 - 51
IEEE Circuits and Systems Magazine - Q2 2021 - 52
IEEE Circuits and Systems Magazine - Q2 2021 - 53
IEEE Circuits and Systems Magazine - Q2 2021 - 54
IEEE Circuits and Systems Magazine - Q2 2021 - 55
IEEE Circuits and Systems Magazine - Q2 2021 - 56
IEEE Circuits and Systems Magazine - Q2 2021 - 57
IEEE Circuits and Systems Magazine - Q2 2021 - 58
IEEE Circuits and Systems Magazine - Q2 2021 - 59
IEEE Circuits and Systems Magazine - Q2 2021 - 60
IEEE Circuits and Systems Magazine - Q2 2021 - 61
IEEE Circuits and Systems Magazine - Q2 2021 - 62
IEEE Circuits and Systems Magazine - Q2 2021 - 63
IEEE Circuits and Systems Magazine - Q2 2021 - 64
IEEE Circuits and Systems Magazine - Q2 2021 - 65
IEEE Circuits and Systems Magazine - Q2 2021 - 66
IEEE Circuits and Systems Magazine - Q2 2021 - 67
IEEE Circuits and Systems Magazine - Q2 2021 - 68
IEEE Circuits and Systems Magazine - Q2 2021 - 69
IEEE Circuits and Systems Magazine - Q2 2021 - 70
IEEE Circuits and Systems Magazine - Q2 2021 - 71
IEEE Circuits and Systems Magazine - Q2 2021 - 72
IEEE Circuits and Systems Magazine - Q2 2021 - 73
IEEE Circuits and Systems Magazine - Q2 2021 - 74
IEEE Circuits and Systems Magazine - Q2 2021 - 75
IEEE Circuits and Systems Magazine - Q2 2021 - 76
IEEE Circuits and Systems Magazine - Q2 2021 - 77
IEEE Circuits and Systems Magazine - Q2 2021 - 78
IEEE Circuits and Systems Magazine - Q2 2021 - 79
IEEE Circuits and Systems Magazine - Q2 2021 - 80
IEEE Circuits and Systems Magazine - Q2 2021 - 81
IEEE Circuits and Systems Magazine - Q2 2021 - 82
IEEE Circuits and Systems Magazine - Q2 2021 - 83
IEEE Circuits and Systems Magazine - Q2 2021 - 84
IEEE Circuits and Systems Magazine - Q2 2021 - 85
IEEE Circuits and Systems Magazine - Q2 2021 - 86
IEEE Circuits and Systems Magazine - Q2 2021 - 87
IEEE Circuits and Systems Magazine - Q2 2021 - 88
IEEE Circuits and Systems Magazine - Q2 2021 - 89
IEEE Circuits and Systems Magazine - Q2 2021 - 90
IEEE Circuits and Systems Magazine - Q2 2021 - 91
IEEE Circuits and Systems Magazine - Q2 2021 - 92
IEEE Circuits and Systems Magazine - Q2 2021 - 93
IEEE Circuits and Systems Magazine - Q2 2021 - 94
IEEE Circuits and Systems Magazine - Q2 2021 - 95
IEEE Circuits and Systems Magazine - Q2 2021 - 96
IEEE Circuits and Systems Magazine - Q2 2021 - 97
IEEE Circuits and Systems Magazine - Q2 2021 - 98
IEEE Circuits and Systems Magazine - Q2 2021 - 99
IEEE Circuits and Systems Magazine - Q2 2021 - 100
IEEE Circuits and Systems Magazine - Q2 2021 - 101
IEEE Circuits and Systems Magazine - Q2 2021 - 102
IEEE Circuits and Systems Magazine - Q2 2021 - 103
IEEE Circuits and Systems Magazine - Q2 2021 - 104
IEEE Circuits and Systems Magazine - Q2 2021 - 105
IEEE Circuits and Systems Magazine - Q2 2021 - 106
IEEE Circuits and Systems Magazine - Q2 2021 - 107
IEEE Circuits and Systems Magazine - Q2 2021 - 108
IEEE Circuits and Systems Magazine - Q2 2021 - 109
IEEE Circuits and Systems Magazine - Q2 2021 - 110
IEEE Circuits and Systems Magazine - Q2 2021 - 111
IEEE Circuits and Systems Magazine - Q2 2021 - 112
IEEE Circuits and Systems Magazine - Q2 2021 - 113
IEEE Circuits and Systems Magazine - Q2 2021 - 114
IEEE Circuits and Systems Magazine - Q2 2021 - 115
IEEE Circuits and Systems Magazine - Q2 2021 - 116
IEEE Circuits and Systems Magazine - Q2 2021 - 117
IEEE Circuits and Systems Magazine - Q2 2021 - 118
IEEE Circuits and Systems Magazine - Q2 2021 - 119
IEEE Circuits and Systems Magazine - Q2 2021 - 120
IEEE Circuits and Systems Magazine - Q2 2021 - 121
IEEE Circuits and Systems Magazine - Q2 2021 - 122
IEEE Circuits and Systems Magazine - Q2 2021 - 123
IEEE Circuits and Systems Magazine - Q2 2021 - 124
IEEE Circuits and Systems Magazine - Q2 2021 - 125
IEEE Circuits and Systems Magazine - Q2 2021 - 126
IEEE Circuits and Systems Magazine - Q2 2021 - 127
IEEE Circuits and Systems Magazine - Q2 2021 - 128
IEEE Circuits and Systems Magazine - Q2 2021 - Cover3
IEEE Circuits and Systems Magazine - Q2 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com