IEEE Computational Intelligence Magazine - August 2021 - 15

multi-objective model compression method is presented. The
model compression problem is formulated as a multi-objective
problem (MOP), which has several objective functions and constraints
[43]. Then, an evolutionary algorithm is adopted to solve
the MOP. The goal of the optimization is to find a set of Pareto
optimal solutions that represents various trade-offs on the
desired objectives, thus enabling the deployment of the AI models
on edge devices with different resource constraints.
A. Problem Formulation
This work aims to compress a well-trained model to achieve
high accuracy, low energy consumption, and low model size.
By providing different pruning amount, p, and quantization
depth, q, the compressed model should result in different accuracy,
energy consumption, and model size. The goal of the optimization
is to reduce the energy consumption or model size
while at the same time making the accuracy of the model as
high as possible. The relationship between the accuracy, the
pruning amount, p, and the quantization depth
[, ,, ]
q qq qL
where f (·)1
= 12 f is denoted as
Accuracy fp q1
= (, ),
(2)
represents the accuracy score of the model
obtained by pruning p of the weight parameters in each layer
of the original model, then quantizing the parameters in the
i-th layer with the depth of qi
bits, and L denotes the number
of layers in the original model.
The energy consumption of the inference is constrained by
the battery's capacitance of edge devices. Exceeding the energy
budget of the edge device will greatly limit the implementation
of AI applications. From the perspective of users, it is
usually acceptable to trade a bit of loss of accuracy for a large
amount of reduction on energy consumption, especially for
edge devices. For a trained model, the energy consumption in
inference is also related to the exact dataflow design d applied
on the edge devices. The relationship among the pruning
amount p, the quantization depth q, and the dataflow design d
is denoted as follows:
Energy fp dq2
= (, ,.
)
(3)
The model size is constrained by the capacities of on-chip
memory modules in edge devices. If the model size exceeds
the limitation, the model inference procedure requires to load
and save weights/features maps through the off-chip memory.
Given the fact that off-chip memory access consumes much
larger energy consumption than the on-chip memory access
[10], the energy consumption of the inference process increases
tremendously. Furthermore, the app stores are sensitive to
the size of the binary files, e.g., App Store has the restriction
" apps above 100 MB will not download until you connect to
Wi-Fi " [10]. Hence, it is important to shrink the size of the
model and to make sure that the entire model can be fit into
the memory constraint of the edge devices. For a given model,
the model size highly depends on the coding scheme c applied
to store the weights. The relationship between the model size,
the pruning amount p, the quantization depth q and the coding
scheme c is defined as
Model Size fp cq3
= (, ,).
(4)
There are L + 3 variables, and L denotes the number of layers
in the original model. The value of the variable p is a real
number that indicates the pruning amount in all the layers of
the model. The value of the variable qi
is an integer that
reflects the quantization depth in the i-th layer of the model.
The constraints on these variables are as follows:
pp p
qq q
dd dd d
c ccc
lu
li u
##
##
!
!
where pl
and pu
ing amount, ql and qu
and c3
,
,
{, ,, },
{ ,,},
1234
12 3
(5)
quantization depth, ,, , and d4
dataflow designs of :, :, :
dd d12 3
are the upper and lower bounds of the prunare
the upper and lower bounds of the
correspond to the four
, and c ,1
XYCC FFIO XY and :XFX
c2
indicate three parameter coding schemes of the normal
coding, COO and CSR, respectively. In this work, the pruning
amount is assumed to be from 0% to 100%, and the quantization
depth of each layer ranges from 1 bit to 23 bits.
Two bi-objective optimization problems are studied. In the
first problem, it explores possible combinations of pruning
amount and quantization depth, and aims to maximize the
model accuracy f1
and minimize the energy consumption ,f2
assuming the dataflow design to be d. Mathematically, the
bi-objective problem can be formulated as following:
'
max
min
fp
fp
1
2
(, ),
(, ,),
q
q
d
s.t. *
pp p
qq q
dd dd d
lu
li u
##
##
!
,
,
{, ,, }.
1234
The second bi-objective problem considers to maximize
the accuracy f1
and minimize the model size f3
'
max
min
fp
fp c
1
3
q
q
(, ),
(, ,),
s.t. *
pp p
qq q
cc cc
lu
li u
12 3
##
##
simultaneously,
assuming the coding scheme to be c, namely, the following
problem:
,
,
! { ,,}.
Note that this work formulates two bi-objective optimization
problems rather than a three-objective optimization problem.
There are two reasons. Firstly, if one optimizes the energy
consumption and the model size simultaneously (i.e., different
dataflow designs and different coding schemes will be considered
at the same time), the decision space will be increased
considerably, making the optimization much harder and consuming
more computation resource. Secondly, as the evaluation
of each individual has a high computational cost, the
population size cannot be a large number. Typically, the population
size is set to be smaller than 100. A three-objective
space will lead to the solution set to be much more sparse
than a bi-objective space.
AUGUST 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 15
(7)
(6)
IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
