IEEE Consumer Electronics Magazine - May/June 2022 - 100

Special Section on ISVLSI-2020
the tile width (Tc) and the data width (W) are the
same, then a single transaction of TcTrdw bytes
can access all Tr rows of the frame. Otherwise,
each row of the tile is accessed using a transaction
of Tcdw bytes.
BWA: OFF-CHIP MEMORY ACCESS
OF CL
CNN accelerators access ifm, ofm, and wts of
the layers from off-chip memory. The number of
bytes accessed from off-chip memory (B)of a
layer is the sum riBiþroBoþrwBw, where Bi, Bo,
and Bw are the number of bytes accessed in one
trip and ri, ro, and rw are the trips count of ifm,
ofm, and wts, respectively. Trips count of each
data depend on the data reuse scheme. It can be
computed using the tile dimensions and layer
shape for a given data reuse scheme.17 Bi, Bo,
and Bw can be computed using the BWA
approach (section " BWA: Off-Chip Memory
Access Computation of 3-D Data " ).
Optimizing the Off-Chip Memory Access
The CNN accelerator stores the tiles of ifm,
ofm, and wts in on-chip memory to perform the
computations. As tiles of all three data types are
stored in on-chip memory, the tile dimensions
can take only values such that the sum of the
sizes of all the three tiles is not greater than the
on-chip memory size. This constraint reduces
the possible values of tile dimensions and
reduces the search space.
Determining the optimal tile dimensions that
minimize B is a constraint optimization problem.
The optimal tile dimensions can be determined
by enumerating all the legal tile dimensions and
determining the memory accesses for each tile.
BWA: OFF-CHIP MEMORY ACCESS
OF FCL
The computations of FCLs are a special case of
CLs with additional constraints on layer shapes
and parameters. The ifm volume is the same as a
filter volume, i.e., Hi ¼Wi ¼K [see Figure 3(b)],
padding P¼0,and stride S¼1. In CNNs, typical values
of Kare 1, 3, 5, 7, and 11. Due to small values of
Hi and Wi, these dimensions are not partitioned.
The number of frames in input and output feature
map tiles are the unknowns to be determined.
100
Determining the optimal tile dimensions for FCLs
is a constraint optimization problem similar to
convolution layers. It can be determined by computing
the off-chipmemory accesses for each legal
tile dimension.
EVALUATION
Implementation
For a given CNN, the optimal tile dimensions
and data reuse scheme need to be determined
once. To determine the optimal tile dimensions,
we have developed an offline tool that computes
the off-chip memory access of CLs and FCLs of
CNN. The tool takes layers shape, bus width,
data bit width, and on-chip memory size as input
and analyzes off-chip memory accesses (B) for
all tile dimensions that satisfy the constraints,
using the BWA approach. It analyzes the off-chip
memory access for different data reuse schemes
and determines each layer's optimal data reuse
scheme and tile dimensions. Running on Intel
Core i7-6700 CPU (@3.40 GHz 8) with 16 GB
RAM, the tool took less than 60 minutes to evaluate
the optimal solution for all the layers of
VGG16. Our approach is orthogonal to the accelerator's
data reuse scheme and does not impact
the computation engine design. The BWA
approach can be used by the CNN accelerators
that support adaptive tiling scheme.
We have also implemented the C++ code to
perform the CL operations using the Xilinx SDSoC
framework, SDx v2018.3. The SDSoC framework
generates the hardware functions from the C++
functions. We have used zero_copy as a data
mover and integrated AXI Performance Monitor
IP21 to log the memory traffic and access latency
of the off-chip memory accesses. Our target platform
was ZedBoard, which uses 64 bits wide AXI
bus to access the off-chip memory (DRAM). The
code can be configured for different tile dimensions
and layer shapes. Using the logs generated
by APM IP, we validated the results computed by
our offline tool and measured the execution time
and memory access latencies of CNN layers.
Benchmarks. We experimented with three
popular CNN networks, AlexNet,22 VGG16,11 and
ResNet12 having 8, 16, and 50 layers, respectively,
with varying layer shapes. To compare
IEEE Consumer Electronics Magazine
IEEE Consumer Electronics Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May/June 2022
