IEEE Computational Intelligence Magazine - August 2021 - 75

10,000 test images. We show the examples of CIFAR10 in
Figure 6. In contrast, CIFAR100 has 100 classes, each with
500 training images and 100 test images. In the experiment, in
order to match the conventions of state-of-the-art CNNs, each
side of each image is filled with four zeros, and then the image
is randomly cropped to its original size. After that, each side is
horizontally flipped randomly, and the images are input finally
in the proposed algorithm.
B. Comparison Algorithms
To verify the effectiveness of the proposed algorithm, many
state-of-the-art algorithms are used for comparison. Ac -
cording to the literature [29], the existing state-of-the-art
CNNs are divided into the following three types: The first
type is purely hand-designed CNNs, which require a lot of
expert knowledge, such as VGG [10], ResNet [5],
DenseNet [12], Network in Network [53], Highway Network
[54], Maxout [55], and ALL-CNN [56]. The purpose
of choosing these hand-designed networks is to prove the
superiority of the automatically searched network architecture.
The second type of network is designed with a
semi-automatic architecture, such as Genetic CNN [28],
Hierarchical Evolution [57], and EAS [58]. The third is a
fully-automatically designed network architecture, such as
Large-scale Evolution [26], NAS [17], MetaQNN [59],
AE-CNN [29], Firefly-CNN [60], EPSO-CNN [61], and
NSGANet [62] which is based on a multi-objective NAS
algorithm [63].
C. Parameter Setting
We compare the existing state-of-the-art algorithms by
consulting the results presented in other papers. The reason
for this is that the algorithms in other papers generally use
the best accuracy, which saves a lot of time. In this paper,
because the proposed algorithm is based on GA architecture
search [29], the population size is set to 20, the crossover
probability is
set
to 0.9, and the three mutation
operation probabilities (adding, removing, and replacing)
are set to 0.4, 0.3, and 0.3, respectively. Following the recommendation
of the machine learning community, the
datasets are randomly divided into one-fifth as validation
datasets. Finally, all classification error rates are obtained on
the same validation datasets.
In this paper, when training the obtained neural network,
Adam is used as the optimizer to learn the weights of the neural
network [43]. The number of epochs is set to 250, the initial
learning rate is set to 0.1, and the batch size is set to 80, the
learning rate is adjusted according to the epoch.
In addition, this paper uses a block-based design method.
Each block and its parameters in the block are set as follows:
The maximum number of RUs, DUs, and PUs is set to three,
and the maximum number of RBs in each RU is set to four.
The selectable parameters in the DB are set to 12, 20, and 40.
The maximum number of convolutional layers is set to five in
the DB. Note that, because the experimental device used in
this paper is one GeForce GTX 2080Ti, and the video memory
is 11 GB, some of the parameters can be adjusted by the
user. Because of the device's video memory limitation, out of
memory error may occur if the parameter setting is too large.
V. Experimental Results and Analysis
This paper evaluates the performance of the algorithms using
not only the classification accuracy, but also the number of
parameters (i.e., model size) and GPU/Days. According to the
concept of GPU/Days proposed by some previous researchers,
the time consumption in the final search process of the algorithm
can be calculated as follows: If n GTX 1080Ti cards are
used to run on the CIFAR10 dataset for m days, the computational
resource consumed by the algorithm on CIFAR10 is
mn) GPU/Days. In addition, the proposed algorithm uses the
standard in [29] in which Sun et al. divided the current stateof-the-art
CNNs into three categories: hand-crafted, semiautomatic,
and completely automatic.
A. Classification Performance of the
Algorithms on CIFAR10
Table II shows the performance comparison of the proposed
algorithm SaMuNet and the comparison algorithms on
CIFAR10 and CIFAR100. The second and third columns represent
the validation classification error rates of different CNNs
on CIFAR10 and CIFAR100, respectively. The fourth column
represents the sizes of the final models. The fifth column represents
the sizes of GPU/Days consumed by each algorithm, and
the sixth column is the constructive method of each model.
The '-' in the table indicates that this indicator has not been
reported in the relevant reference.
As shown in Table II, compared with the purely hand-crafted
model, which includes ResNet and DenseNet, the classification
accuracy of SaMuNet on CIFAR10 has achieved the
second lowest error rate, which is a little higher than FireflyCNN.
However, the number of parameters from the final
model which is found by our algorithm is much lower than
Firefly-CNN. The proposed algorithm has an error rate 1.6%
Airplane
Automobile
Horse
FIGURE 6 Three examples from CIFAR10.
AUGUST 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 75
IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
