IEEE Computational Intelligence Magazine - August 2021 - 13

C. Coding Scheme for the
Parameters
After pruning, the filter in the model
becomes a sparse matrix, which means
that it contains plenty of zero elements.
To store the sparse matrix into the
memory, many coding schemes are
developed, and the choice of the coding
scheme mainly depends on the
characteristics of the matrix. In this
work, three coding schemes are considered.
The first one is the normal coding
scheme, where it stores those zero elements
in the same way as those nonzero
elements. In other words, it keeps
the space for all the zero elements in
the matrix. Therefore, the storage size of
the normal coding scheme is
Nq ,i
$
where N is the total number of weight
elements in the matrix and qi
W2
is the
quantization depth of the weight.
In some cases, it is a waste of the
O
Dataflow - Fx:FY
memory capacity to store all these zero
weights. For saving the memory space,
various new coding schemes are proposed
to shrink the size of the sparse
matrix. One common coding scheme is
called the Coordinate (COO) coding.
In the COO coding scheme, the non-zero elements are stored,
along with the row index and the column index, and zero elements
are ignored. Another popular coding scheme is called the
compressed sparse row (CSR) coding. In the CSR coding
scheme, only the values of the non-zero elements are stored
along with the column index and the row offset. To further
save the memory space, one version of the CSR coding
scheme only stores the relative distance between two non-zero
elements, which is shown in Figure 2. For this example, the
weight matrix has eight elements, and three of them are nonzero
elements. Only the values of these three elements and
their relative positions in the array are stored. The second nonzero
element is three slots away from the first non-zero element,
and the integer 2 is recorded as a relative row index for
the second element. It is assumed that each non-zero element
requires three bits to store the relative row index. If one nonzero
element is far away from the previous element, zero-padding
elements are inserted into the array to avoid overflow.
Therefore, the storage size of the CSR coding scheme with
relative positions is ·(
zero elements and qi
np )3i + , where n is the number of nonis
the quantization depth of the weight.
XXX 00 XXX 00 0
III. Related Work
W0 W1 W2 W3 W4 W5 W6 W7
A. Model Compression
Model compression aims to compress and accelerate DNN
models. Different approaches target different objectives, such as
FIGURE 2 The CSR coding scheme with relative positions, which
stores the values of non-zero elements and their relative positions in
the array.
XXX
Processing Element
kkk
Register File
Dataflow - X:Fx
Data Direction
FIGURE 1 Examples of the four popular dataflow designs [15]. For simplicity, only four PEs are
shown in each example. /,
WW Ik, and Ok are the elements in the weight, the input feature
map, and the output feature map, respectively.
model size, number of floating-point operations per second
(FLOPs), latency, and energy efficiency. The initial intention of
model compression is to alleviate the on-chip storage limit for
complicated CNN models [10]. Since then, many approaches
have been proposed to shrink the model size of CNNs [23],
[24]. There are two major branches in this area. The first branch
focuses on the computation cost, and they target the number of
FLOPs [25]. For example, Li et al. [26] proposed to prune
whole filters from CNNs, avoiding sparse connectivity patterns
and reducing the computational cost significantly. Lemaire et al.
proposed a budgeted regularized pruning framework for deep
CNNs [25], which makes the compressed model less computation-intensive.
The second branch targets the inference speed
[27], [28]. For instance, He et al. leveraged reinforcement
Distance = 2
NA
2
Distance = 3
3
W3
W1
O0
O1
I0
I1
W0
I2
W2
O2
O0
W1
I3
W3
Dataflow - X:Y
I0
W0
I2
I1
W1
I3
I0
W0
I1
I2
O3
W1
O0
Dataflow - Cl:Co
I1
W11
O1
O1
I0
W0
W10
AUGUST 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 13

IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021