IEEE Computational Intelligence Magazine - August 2021 - 17

only prunes the model. Specifically, assuming t
as granularity, it prunes the model by
100 /t
times. At the i-th time step, it starts from the
well-trained model and prunes the model
gradually using the multi-step pruning method,
until the target pruning amount reaches
.
Pruning and quantization are two popular techniques
in DNN model compression, which show substantial
improvement in energy efficiency.
i $ t After that, it saves the compressed model into a pre-pruned
models library. In the second stage, it loads one of the prepruned
models from the library based on the required pruning
amount p, and then quantizes the parameters on the pre-pruned
model based on the required quantization depth q. Since pruning
and quantization are two orthogonal operations, the final
compressed model will be equivalent to the compressed model
that is pruned and quantized at the same time. Lastly, it obtains
the accuracy by performing the model inference and read the
energy consumption from an energy estimator.
The proposed approach can efficiently speed up the optimization
process. To obtain the accuracy and energy consumption under
a given pruning amount and quantization depth, it does not need
to fine-tune the model anymore. Before the optimization process, it
completes the procedures in stage-I and saves only 100 compressed
models into the library, assuming 1% granularity. The number of
saved models is much less than 100 23L
# , i.e., the number of possible
compressed models in the whole exploration space. For each
combination of pruning amount and quantization depth, the time
cost of evaluating the individual is roughly equal to the inference
time cost of the model.
V. Experimental Results and Analysis
The proposed method is evaluated on three baseline CNN
models: MobileNet [46], VGG-16 [2] and LeNet-5 [47], which
have different characteristics. MobileNet is a neural network
specially designed for mobile and embedded vision applications.
VGG is a typical deep neural network, which was in the
X:Y (3.4)
0.2
0.4
0.6
0.8
1
0.7
0.75
0.8
Accuracy
MobileNet
0.85
0.9
CI:CO (3.2)
FX:FY (4.6) X:FX (4.7)
2.5
1.5
2
0.5
1
0.75
0.8
0.85
Accuracy
VGG-16
0.9
0.95
first place on the image localization and the second place on
the image classification task in the ImageNet Large-Scale Visual
Recognition Challenge (ILSVRC) in 2014. LeNet-5 is a simple
network for handwritten and machine-printed character
recognition. It consists of only two sets of convolutional and
average pooling layers, followed by a flattening convolutional
layer, two fully-connected layers, and a Softmax classifier.
MobileNet and VGG-16 are tested for color image classification
on the CIFAR-10 dataset [42], and LeNet-5 is applied to
recognize handwriting digits in the MNIST dataset [47].
A. Experimental Setting
The NSGA-II algorithm in the python-based tool Pymoo [43]
is used to solve the formulated multi-objective problem. The
neural network is implemented in PyTorch4. During the network
training, the initial learning rate is set to be 0.01, and it
decays by half every 30 epochs. The batch size is set to be 256.
During the multi-objective optimization process, the population
size is set to be 40, and it runs 250 generations in each
execution. The multi-objective optimization and network
training are performed on an NVIDIA Titan Xp graphics
processing unit (GPU) card. Four dataflow designs are considered
as they are the most commonly used dataflow designs
:, :, :,
XYCC FFIO XY and :. The resource requirement is
XFX
calculated based on the Xilinx Virtex UltraScale FPGA and the
energy consumption from the Xilinx XPE toolkit [48]. In the
4PyTorch Open Source Toolkit at https://github.com/pytorch/pytorch.
X:Y (23.1) CI:CO (21.2)
FX:FY (24) X:FX (22.8)
X:Y (31.2)
1
2
3
4
5
6
0.8
0.85
0.9
Accuracy
LeNet-5
FIGURE 5 The solution sets obtained from the bi-objective optimization of accuracy and energy consumption on CIFAR-10 (MobileNet and VGG16)
and MNIST (LeNet-5). The four different dataflow designs are marked with different colors. In the legends, the quoted number after the dataflow
design indicates its energy consumption (mJ) on the original model before the model compression.
AUGUST 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 17
0.95
1
CI:CO (33.1)
FX:FY (31.5) X:FX (28.1)
Energy Consumption (mJ)
Energy Consumption (mJ)
Energy Consumption (mJ)

https://www.github.com/pytorch/pytorch

IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021