IEEE Computational Intelligence Magazine - August 2021 - 71

B. Encoding Strategy
Encoding is very important when solving
practical problems, and different problems
require different coding strategies. For the
EC technique, encoding strategies are generally
divided into two types: direct encoding
and indirect encoding. Direct encoding
directly encodes the precise information into
each individual. Indirect encoding is different from directly
representing the weight or connection of the network, and it
usually represents each individual through a new generation
rule [16]. This encoding method is widely used in neuroevolution
[44]. Because of the increasing number of network layers
in CNNs, the number of parameters that need to be optimized
has also increased exponentially. The backpropagation
algorithm [15] has been proven to be superior in optimizing
weights and it is directly applied to the weight optimization of
the CNN. Thus, it is not necessary to consider the optimization
of the weights of CNN, and only some hyperparameters
need to be optimized.
To make operators such as crossover and mutation easier to
implement and maintain the flexibility of CNNs' architectures,
this paper uses a direct variable-length encoding strategy to
represent CNN architecture. This encoding strategy uses basic
units RU, DU and PU. For the RU, a random number is used
to represent the amount of RBs, and each RB contains at least
one 3 × 3 convolution kernel. Each RB also includes a skip
connection operation.
For the DU, a random number is used to represent the number
of DBs, and each DB also contains at least one 3 × 3 convolution
kernel. The difference between DU and RU is that the skip
connections in the DB are densely connected (i.e., the input of
each layer contains all the previous layer outputs). Meanwhile,
each PU includes only one pooling layer of either type: average
pooling layer and maximum pooling layer. In summary, RUs,
DUs and PUs are composed of RBs, DBs and pooling layers,
respectively. For different RBs, the unknown parameters are the
number of input and output channels. For different DBs, in addition
to the number of input and output channels, the unknown
parameters include a growth rate k (to calculate the number of
input channels for each layer). To express individual information
and make the genetic operator more flexible, this article uses the
variable-length encoding strategy [15]. As shown in Figure 4, an
individual contains RUs, DUs, and PUs. In the later stage of the
evolution process, the block-based method in the figure is used
as the basic unit.
C. Initialization of the Population
Population initialization is the first stage in the evolution process,
and it involves a random initialization method that obeys a uniform
distribution for each individual. As described in Part A,
each individual represents a CNN architecture. To find the best
individual, GA with self-adaptive mutation is used in the evolution
process. The proposed algorithm uses a block-based design
method, which is different from traditional NAS methods that
In this paper, each individual represents the architecture
of one CNN, and the fitness is the classification
accuracy of each CNN on the validation dataset. The
higher the classification accuracy, the higher the fitness.
use layers as basic units. Inspired by the success of ResNet [5]
and DenseNet [12], in this paper, RUs, DUs, and PUs are used as
the basic units. RUs are composed of RBs, DUs are composed
of DBs, and PUs are composed of pooling layers. The proposed
algorithm does not optimize the fully-connected layer. The main
reason is that the neurons will cause overfitting due to the characteristics
of the fully-connected layers [45]. To alleviate this
phenomenon, operations such as dropout [46] need to be used,
but this will increase the number of parameters that need to be
optimized and increase the number of computing resources.
Algorithm 1 Framework of SaMuNet.
Input: The population size (N), the number of maximal iteration
(nIter), the crossover probability (m), the mutation probability
(μ), the current iteration (t).
Output: The best individual with the architecture of
the CNN.
1: Randomly initialize population P0;
2: Evaluate each individual;
3: while t < nIter do
4: Qt ! {};
5:
6:
while Q N


IEEE Computational Intelligence Magazine - August 2021
Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
Contents


IEEE Computational Intelligence Magazine - August 2021 - Cover1

IEEE Computational Intelligence Magazine - August 2021 - Cover2

IEEE Computational Intelligence Magazine - August 2021 - Contents

IEEE Computational Intelligence Magazine - August 2021 - 2

IEEE Computational Intelligence Magazine - August 2021 - 3

IEEE Computational Intelligence Magazine - August 2021 - 4

IEEE Computational Intelligence Magazine - August 2021 - 5

IEEE Computational Intelligence Magazine - August 2021 - 6

IEEE Computational Intelligence Magazine - August 2021 - 7

IEEE Computational Intelligence Magazine - August 2021 - 8

IEEE Computational Intelligence Magazine - August 2021 - 9

IEEE Computational Intelligence Magazine - August 2021 - 10

IEEE Computational Intelligence Magazine - August 2021 - 11

IEEE Computational Intelligence Magazine - August 2021 - 12

IEEE Computational Intelligence Magazine - August 2021 - 13

IEEE Computational Intelligence Magazine - August 2021 - 14

IEEE Computational Intelligence Magazine - August 2021 - 15

IEEE Computational Intelligence Magazine - August 2021 - 16

IEEE Computational Intelligence Magazine - August 2021 - 17

IEEE Computational Intelligence Magazine - August 2021 - 18

IEEE Computational Intelligence Magazine - August 2021 - 19

IEEE Computational Intelligence Magazine - August 2021 - 20

IEEE Computational Intelligence Magazine - August 2021 - 21

IEEE Computational Intelligence Magazine - August 2021 - 22

IEEE Computational Intelligence Magazine - August 2021 - 23

IEEE Computational Intelligence Magazine - August 2021 - 24

IEEE Computational Intelligence Magazine - August 2021 - 25

IEEE Computational Intelligence Magazine - August 2021 - 26

IEEE Computational Intelligence Magazine - August 2021 - 27

IEEE Computational Intelligence Magazine - August 2021 - 28

IEEE Computational Intelligence Magazine - August 2021 - 29

IEEE Computational Intelligence Magazine - August 2021 - 30

IEEE Computational Intelligence Magazine - August 2021 - 31

IEEE Computational Intelligence Magazine - August 2021 - 32

IEEE Computational Intelligence Magazine - August 2021 - 33

IEEE Computational Intelligence Magazine - August 2021 - 34

IEEE Computational Intelligence Magazine - August 2021 - 35

IEEE Computational Intelligence Magazine - August 2021 - 36

IEEE Computational Intelligence Magazine - August 2021 - 37

IEEE Computational Intelligence Magazine - August 2021 - 38

IEEE Computational Intelligence Magazine - August 2021 - 39

IEEE Computational Intelligence Magazine - August 2021 - 40

IEEE Computational Intelligence Magazine - August 2021 - 41

IEEE Computational Intelligence Magazine - August 2021 - 42

IEEE Computational Intelligence Magazine - August 2021 - 43

IEEE Computational Intelligence Magazine - August 2021 - 44

IEEE Computational Intelligence Magazine - August 2021 - 45

IEEE Computational Intelligence Magazine - August 2021 - 46

IEEE Computational Intelligence Magazine - August 2021 - 47

IEEE Computational Intelligence Magazine - August 2021 - 48

IEEE Computational Intelligence Magazine - August 2021 - 49

IEEE Computational Intelligence Magazine - August 2021 - 50

IEEE Computational Intelligence Magazine - August 2021 - 51

IEEE Computational Intelligence Magazine - August 2021 - 52

IEEE Computational Intelligence Magazine - August 2021 - 53

IEEE Computational Intelligence Magazine - August 2021 - 54

IEEE Computational Intelligence Magazine - August 2021 - 55

IEEE Computational Intelligence Magazine - August 2021 - 56

IEEE Computational Intelligence Magazine - August 2021 - 57

IEEE Computational Intelligence Magazine - August 2021 - 58

IEEE Computational Intelligence Magazine - August 2021 - 59

IEEE Computational Intelligence Magazine - August 2021 - 60

IEEE Computational Intelligence Magazine - August 2021 - 61

IEEE Computational Intelligence Magazine - August 2021 - 62

IEEE Computational Intelligence Magazine - August 2021 - 63

IEEE Computational Intelligence Magazine - August 2021 - 64

IEEE Computational Intelligence Magazine - August 2021 - 65

IEEE Computational Intelligence Magazine - August 2021 - 66

IEEE Computational Intelligence Magazine - August 2021 - 67

IEEE Computational Intelligence Magazine - August 2021 - 68

IEEE Computational Intelligence Magazine - August 2021 - 69

IEEE Computational Intelligence Magazine - August 2021 - 70

IEEE Computational Intelligence Magazine - August 2021 - 71

IEEE Computational Intelligence Magazine - August 2021 - 72

IEEE Computational Intelligence Magazine - August 2021 - 73

IEEE Computational Intelligence Magazine - August 2021 - 74

IEEE Computational Intelligence Magazine - August 2021 - 75

IEEE Computational Intelligence Magazine - August 2021 - 76

IEEE Computational Intelligence Magazine - August 2021 - 77

IEEE Computational Intelligence Magazine - August 2021 - 78

IEEE Computational Intelligence Magazine - August 2021 - 79

IEEE Computational Intelligence Magazine - August 2021 - 80

IEEE Computational Intelligence Magazine - August 2021 - 81

IEEE Computational Intelligence Magazine - August 2021 - 82

IEEE Computational Intelligence Magazine - August 2021 - 83

IEEE Computational Intelligence Magazine - August 2021 - 84

IEEE Computational Intelligence Magazine - August 2021 - 85

IEEE Computational Intelligence Magazine - August 2021 - 86

IEEE Computational Intelligence Magazine - August 2021 - 87

IEEE Computational Intelligence Magazine - August 2021 - 88

IEEE Computational Intelligence Magazine - August 2021 - 89

IEEE Computational Intelligence Magazine - August 2021 - 90

IEEE Computational Intelligence Magazine - August 2021 - 91

IEEE Computational Intelligence Magazine - August 2021 - 92

IEEE Computational Intelligence Magazine - August 2021 - 93

IEEE Computational Intelligence Magazine - August 2021 - 94

IEEE Computational Intelligence Magazine - August 2021 - 95

IEEE Computational Intelligence Magazine - August 2021 - 96

IEEE Computational Intelligence Magazine - August 2021 - Cover3

IEEE Computational Intelligence Magazine - August 2021 - Cover4


https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12


https://www.nxtbookmedia.com