IEEE Computational Intelligence Magazine - May 2023 - 91

TABLE V Comparison of different network architectures on ImageNet-16-120.
ARCHITECTURE
ENAS [30]
AmoebaNet-A [7]
DARTS [28]
P-DARTS [60]
PC-DARTS [40]
GDAS [27]
EX-Net(CIFAR-10)(ours)
EX-Net(CIFAR-100)(ours)
4.6
3.2
3.3
5.3
6.2
2.5
2.0
4.3
0.5
3150
1.5
0.3
0.1
0.3
0.02
0.02
TESTACCURACY(%) PARAMETERS(M) GPU DAYS GPUs
16.32
46.21
16.32
45.24
45.53
42.21
44.56
46.87
1
450
1
1
1
1
1
1
shows competitive test accuracy while
exhibiting an advantage in terms of the
number ofparameters. Furthermore, EXNet
greatly reduces the search time cost
and the required computational resource
consumption.
3) Comparison With Automatic
NASMethods
Compared to the competing completely
automatic NAS algorithms, EXNet
exhibits great superiority over
Large-scale Evolution and NAS in
terms of the accuracy and number of
parameters. In addition, EX-Net consumes
only 0.02 GPU Days, which is
substantially less than Large-scale Evolution
and NAS. Additionally, the
GPU resources required by EX-Net
are 800x less than those required by
NAS. EX-Net has better test accuracy
and fewer parameters than AmoebaNet-A.
The GPU Days required by
EX-Net is only 0.02, which is 1/
157,500 of that needed by AmoebaNet-A,
and the computational resources
required by the GPU are just 1/450
of those demanded by AmoebaNet-A.
EX-Net is better than AE-CNN in
terms of the mean and deviation of test
accuracy and the number of parameters
on CIFAR-10 and CIFAR-100. EXNet
can obtain a better improvement
in the search time cost and the required
GPU resource consumption. Compared
to CNN-GA, EX-Net has a higher
test accuracy on CIFAR-10 and has
fewer parameters. In addition, EX-Net
achieves better accuracy on the more
complex CIFAR-100 dataset, while the
number of parameters is close to that of
CNN-GA. The search time ofEX-Net
is approximately 1/1750 of that consumed
by CNN-GA. NSGA-Net
attains slightly better accuracy than EXNet
on CIFAR-10 (97.5% vs. 96.95%),
but EX-Net has smaller deviations of
test accuracy (0.07 vs. 0.10), and only
consumes 1/13 of the parameters
required by NSGA-Net (2.0M vs.
26.8M). When using the same computational
resources, the search time of
EX-Net is 200x less than that of
NSGA-Net. Compared to LF-MOGP,
EX-Net can achieve a significant advantage
in terms of accuracy on both
datasets. Moreover, EX-Net only consumes
0.02 GPU Days, which is 500x
less than LF-MOGP. Therefore, EXNet
shows significant advantages over
automatic algorithms in all objectives.
4)Discussion
In summary, EX-Net outperforms most
manually-designed network architectures
in terms of the test accuracy while
requiring fewer parameters. EX-Net
also shows excellent advantages over
most of the automatic NAS algorithms
regarding test accuracy and the number
of parameters. The proposed approach
also requires fewer GPU resources and
achieves 200x to 1,120,000x search
time reductions. Compared to the semiautomatic
NAS algorithms, the test
accuracy advantage of EX-Net is not
apparent since other algorithms involve
manual fine-tuning. However, EX-Net
can achieve more minor deviations in
test accuracy, which demonstrates the
robustness and stability of the EX-Net.
Furthermore, EX-Net utilizes fewer
parameters, and the search time cost and
computational resource consumption
are significantly reduced, which is the
primary purpose of the work in this
paper.
Table II shows that each run by
LoNAS has a similar search time on
CIFAR-10 and CIFAR-100, demonstrating
the robustness of the proposed
algorithm regarding the time cost. By
comprehensively considering the test
accuracies and the number of parameters,
Tables III and IV show the best
network architectures discovered by the
proposed algorithm on the CIFAR-10
and CIFAR-100 datasets. They also
show that the best network architecture
obtained on CIFAR-10 is composed of
three Reg Units with 34 convolutional
layers. The network FLOPs are 0.91 G.
The best network architecture obtained
on CIFAR-100 comprises four Reg
Units of 46 convolutional layers, and
the FLOPs ofthe network are 1.05 G.
E. Transferability
The ImageNet-16-120 dataset [65] is
also considered for investigating the
transferability of the network architectures
searched by LoNAS on the
CIFAR-10 and CIFAR-100 datasets.
The same training settings as mentioned
in Section IV-A are used. The comparison
results in Table V show that the network
architecture searched on CIFAR10
obtains better test accuracy than most
manually-designed and automatic architectures.
The network architecture
searched on CIFAR-100 achieves the
best test accuracy on ImageNet-16-120
while consuming minimal GPU Days
and GPU resources. The results indicate
that LoNAS has good transferability to
other datasets.
V. Conclusion
This paper proposes a LoNAS method
that can quickly search for network
architectures with high accuracy and
few parameters. The computational
resources required by LoNAS to find
the network architectures are also low.
In LoNAS, a Reg Block is proposed
based on group convolution and the
SENet module, which helps improve
the accuracy while reducing the number
of parameters. A variable-architecture
MAY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 91
IEEE Computational Intelligence Magazine - May 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2023
