IEEE Computational Intelligence Magazine - August 2021 - 18

implementation, the multipliers and adders are implemented
on LUTs (lookup tables). An MN# multiplier requires
/( )
21# + LUTs [49]. To save the memory space, there is
no need to keep the feature map in local memory after the
computation of each layer. Hence, the size of the local memory
modules must support the weights in all layers and the temporary
feature maps. The pruning and quantization approaches are
described in [10]. For pruning, the 1
MN
, -norm based unstructured
pruning method is adopted and a mask is added to filter out
the pruned weight. For quantization, the linear (uniform)
quantization method is adopted and a scaling factor is used to
lower the precision of the weights.
B. Bi-Objective Optimization of
Accuracy and Energy Consumption
Due to the proposed two-stage pruning and quantization cooptimization
method, one can complete the model compression
and obtain the solution set efficiently. The entire
optimization process includes two stages. The first stage is for
the pre-processing, which takes around 24 hours. The second
stage is for the multi-objective optimization. The solver can
generate optimal solutions within one hour by using a single
NVIDIA Titan Xp graphics processing unit (GPU) card.
Figure 5 shows the solution sets obtained from the bi-objective
optimization of accuracy and energy consumption,
under the four different dataflow designs. Each point in the
figure corresponds to one compressed model in the solution
set obtained by the bi-objective optimization. From the
results, one can see that:
❏ The points marked in different colors cover a large range of
accuracy scores and energy consumption, which means that
EMOMC obtains a solution set with a high diversity for
the model compression of the three baseline CNN models,
under the four dataflow designs. For example, under the
dataflow design of : ,XY the accuracy scores of MobileNet
range from around 75% to 90%, and the energy consumption
from around 0.2 mJ to 0.58 mJ. It offers the right
trade-offs between the two objectives for meeting the constraints
of various edge devices.
❏ From the perspective of energy consumption, if searching
solutions from the one with the highest energy consumption
to the one with the lowest energy consumption, the loss on
accuracy is negligible at the first few points. For instance,
under the dataflow design of : ,XY the energy consumption
of VGG-16 decreases from around 2.3 mJ to 0.5 mJ with an
accuracy drop less than 2%. However, after a certain threshold,
the accuracy loss becomes extremely large. By considering
the model's accuracy, if searching for solutions from the
one with the highest accuracy to the one with the lowest
accuracy, the reduction of energy consumption is remarkable
at the first few points. However, after a certain threshold, the
energy consumption becomes relatively stable.
❏ Different models prefer different dataflow designs. Specifically,
CC
:IO achieves the highest energy efficiency among
the four dataflow designs for MobileNet. However, it is
inferior to other dataflow designs for VGG-16. The reason is
that the convolution layers of different models have different
shapes. In addition to energy consumption, the latency and
cost of edge devices also depend on dataflow designs. The
selection of dataflow designs involves many factors, which
makes it very difficult in practice. This work explores the
optimization results on the four popular dataflow designs.
C. Bi-Objective Optimization of Accuracy and Model Size
Figure 6 demonstrates the solution sets obtained from the biobjective
optimization of accuracy and model size, under three
COO CSR-Relative
Normal (12.5)
COO CSR-Relative
Normal (59.5)
7.5
4.5
6
1.5
3
0.7
0.75
0.8
Accuracy
MobileNet
0.85
0.9
10
15
20
25
5
0.75
0.8
0.85
Accuracy
VGG-16
0.9
0.95
100
125
25
50
75
0.8
0.85
0.9
Accuracy
LeNet-5
FIGURE 6 The solution sets obtained from the bi-objective optimization of accuracy and model size on CIFAR-10 (MobileNet and VGG-16) and
MNIST (LeNet-5). The three different coding schemes are marked with different colors. In the legends, the quoted number after normal coding
scheme indicates the size of the original model before the model compression.
0.95
1
COO CSR-Relative
Normal (240)
18 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2021
Model Size (MBytes)
Model Size (MBytes)
Model Size (MBytes)

IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021