IEEE Computational Intelligence Magazine - August 2021 - 10
Evolutionary Multi-Objective
Model Compression
for Deep Neural Networks
Zhehui Wang and Tao Luo
Agency for Science, Technology and Research (A*STAR),
SINGAPORE
Miqing Li,
University of Birmingham, UK
Joey Tianyi Zhou, Rick Siow Mong Goh,
and Liangli Zhen
Agency for Science, Technology and Research (A*STAR),
SINGAPORE
Abstract-While deep neural networks (DNNs) deliver state-ofthe-art
accuracy on various applications from face recognition to
language translation, it comes at the cost of high computational
and space complexity, hindering their deployment on edge
devices. To enable efficient processing of DNNs in inference, a
novel approach, called Evolutionary Multi-Objective Model
Digital Object Identifier 10.1109/MCI.2021.3084393
Date of current version: 15 July 2021
10 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2021
Compression (EMOMC), is proposed to optimize energy efficiency
(or model size) and accuracy simultaneously. Specifically,
the network pruning and quantization space are explored and
exploited by using architecture population evolution. Furthermore,
by taking advantage of the orthogonality between pruning
and quantization, a two-stage pruning and quantization co-optimization
strategy is developed, which considerably reduces time
cost of the architecture search. Lastly, different dataflow designs
and parameter coding schemes are considered in the optimization
process since they have a significant impact on energy consumption
and the model size. Owing to the cooperation of the
evolution between different architectures in the population, a set
of compact DNNs that offer trade-offs on different objectives
(e.g., accuracy, energy efficiency and model size) can be obtained
in a single run. Unlike most existing approaches designed to
reduce the size of weight parameters with no significant loss of
accuracy, the proposed method aims to achieve a trade-off
between desirable objectives, for meeting different requirements
of various edge devices. Experimental results demonstrate that the
proposed approach can obtain a diverse population of compact
Corresponding author: Liangli Zhen (e-mail: zhenll@ihpc.a-star.edu.sg).
1556-603X/21©2021IEEE
©SHUTTERSTOCK.COM/DMITRIY PRAYZEL
http://www.SHUTTERSTOCK.COM/DMITRIY
IEEE Computational Intelligence Magazine - August 2021
Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
Contents
IEEE Computational Intelligence Magazine - August 2021 - Cover1
IEEE Computational Intelligence Magazine - August 2021 - Cover2
IEEE Computational Intelligence Magazine - August 2021 - Contents
IEEE Computational Intelligence Magazine - August 2021 - 2
IEEE Computational Intelligence Magazine - August 2021 - 3
IEEE Computational Intelligence Magazine - August 2021 - 4
IEEE Computational Intelligence Magazine - August 2021 - 5
IEEE Computational Intelligence Magazine - August 2021 - 6
IEEE Computational Intelligence Magazine - August 2021 - 7
IEEE Computational Intelligence Magazine - August 2021 - 8
IEEE Computational Intelligence Magazine - August 2021 - 9
IEEE Computational Intelligence Magazine - August 2021 - 10
IEEE Computational Intelligence Magazine - August 2021 - 11
IEEE Computational Intelligence Magazine - August 2021 - 12
IEEE Computational Intelligence Magazine - August 2021 - 13
IEEE Computational Intelligence Magazine - August 2021 - 14
IEEE Computational Intelligence Magazine - August 2021 - 15
IEEE Computational Intelligence Magazine - August 2021 - 16
IEEE Computational Intelligence Magazine - August 2021 - 17
IEEE Computational Intelligence Magazine - August 2021 - 18
IEEE Computational Intelligence Magazine - August 2021 - 19
IEEE Computational Intelligence Magazine - August 2021 - 20
IEEE Computational Intelligence Magazine - August 2021 - 21
IEEE Computational Intelligence Magazine - August 2021 - 22
IEEE Computational Intelligence Magazine - August 2021 - 23
IEEE Computational Intelligence Magazine - August 2021 - 24
IEEE Computational Intelligence Magazine - August 2021 - 25
IEEE Computational Intelligence Magazine - August 2021 - 26
IEEE Computational Intelligence Magazine - August 2021 - 27
IEEE Computational Intelligence Magazine - August 2021 - 28
IEEE Computational Intelligence Magazine - August 2021 - 29
IEEE Computational Intelligence Magazine - August 2021 - 30
IEEE Computational Intelligence Magazine - August 2021 - 31
IEEE Computational Intelligence Magazine - August 2021 - 32
IEEE Computational Intelligence Magazine - August 2021 - 33
IEEE Computational Intelligence Magazine - August 2021 - 34
IEEE Computational Intelligence Magazine - August 2021 - 35
IEEE Computational Intelligence Magazine - August 2021 - 36
IEEE Computational Intelligence Magazine - August 2021 - 37
IEEE Computational Intelligence Magazine - August 2021 - 38
IEEE Computational Intelligence Magazine - August 2021 - 39
IEEE Computational Intelligence Magazine - August 2021 - 40
IEEE Computational Intelligence Magazine - August 2021 - 41
IEEE Computational Intelligence Magazine - August 2021 - 42
IEEE Computational Intelligence Magazine - August 2021 - 43
IEEE Computational Intelligence Magazine - August 2021 - 44
IEEE Computational Intelligence Magazine - August 2021 - 45
IEEE Computational Intelligence Magazine - August 2021 - 46
IEEE Computational Intelligence Magazine - August 2021 - 47
IEEE Computational Intelligence Magazine - August 2021 - 48
IEEE Computational Intelligence Magazine - August 2021 - 49
IEEE Computational Intelligence Magazine - August 2021 - 50
IEEE Computational Intelligence Magazine - August 2021 - 51
IEEE Computational Intelligence Magazine - August 2021 - 52
IEEE Computational Intelligence Magazine - August 2021 - 53
IEEE Computational Intelligence Magazine - August 2021 - 54
IEEE Computational Intelligence Magazine - August 2021 - 55
IEEE Computational Intelligence Magazine - August 2021 - 56
IEEE Computational Intelligence Magazine - August 2021 - 57
IEEE Computational Intelligence Magazine - August 2021 - 58
IEEE Computational Intelligence Magazine - August 2021 - 59
IEEE Computational Intelligence Magazine - August 2021 - 60
IEEE Computational Intelligence Magazine - August 2021 - 61
IEEE Computational Intelligence Magazine - August 2021 - 62
IEEE Computational Intelligence Magazine - August 2021 - 63
IEEE Computational Intelligence Magazine - August 2021 - 64
IEEE Computational Intelligence Magazine - August 2021 - 65
IEEE Computational Intelligence Magazine - August 2021 - 66
IEEE Computational Intelligence Magazine - August 2021 - 67
IEEE Computational Intelligence Magazine - August 2021 - 68
IEEE Computational Intelligence Magazine - August 2021 - 69
IEEE Computational Intelligence Magazine - August 2021 - 70
IEEE Computational Intelligence Magazine - August 2021 - 71
IEEE Computational Intelligence Magazine - August 2021 - 72
IEEE Computational Intelligence Magazine - August 2021 - 73
IEEE Computational Intelligence Magazine - August 2021 - 74
IEEE Computational Intelligence Magazine - August 2021 - 75
IEEE Computational Intelligence Magazine - August 2021 - 76
IEEE Computational Intelligence Magazine - August 2021 - 77
IEEE Computational Intelligence Magazine - August 2021 - 78
IEEE Computational Intelligence Magazine - August 2021 - 79
IEEE Computational Intelligence Magazine - August 2021 - 80
IEEE Computational Intelligence Magazine - August 2021 - 81
IEEE Computational Intelligence Magazine - August 2021 - 82
IEEE Computational Intelligence Magazine - August 2021 - 83
IEEE Computational Intelligence Magazine - August 2021 - 84
IEEE Computational Intelligence Magazine - August 2021 - 85
IEEE Computational Intelligence Magazine - August 2021 - 86
IEEE Computational Intelligence Magazine - August 2021 - 87
IEEE Computational Intelligence Magazine - August 2021 - 88
IEEE Computational Intelligence Magazine - August 2021 - 89
IEEE Computational Intelligence Magazine - August 2021 - 90
IEEE Computational Intelligence Magazine - August 2021 - 91
IEEE Computational Intelligence Magazine - August 2021 - 92
IEEE Computational Intelligence Magazine - August 2021 - 93
IEEE Computational Intelligence Magazine - August 2021 - 94
IEEE Computational Intelligence Magazine - August 2021 - 95
IEEE Computational Intelligence Magazine - August 2021 - 96
IEEE Computational Intelligence Magazine - August 2021 - Cover3
IEEE Computational Intelligence Magazine - August 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com