IEEE Computational Intelligence Magazine - May 2023 - 92

encoding strategy based on the Reg
Block is designed to construct an
expanded search space. A training-free
proxy is proposed to evaluate individuals
based on the NTK, reducing the time
and computational resource costs. Furthermore,
a three-stage EA based on
multiple-criteria environmental selection
is designed to balance the exploration
and exploitation of the proposed
algorithm. A set ofmutation operators is
proposed and applied to the Reg Block
to explore more ofthe search space.
The proposed algorithm is examined
on two representative benchmark datasets,
CIFAR-10 and CIFAR-100, and compared
with various state-of-the-art algorithms,
including manually-designed
network architectures, semiautomatic network
architectures, and automatic network
architectures. The experimental
results show that the network architectures
found by LoNAS outperformmost manually-designed
and automatic architectures
in terms of classification performance and
the number of parameters. The network
architectures also achieve the competitive
performance on the semiautomatic architectures
in terms of test accuracy while
improving most algorithms in the number
ofrequired parameters. More importantly,
LoNAS shows significant advantages in
reducing the search time cost and computational
resource consumption. Finally,
the architectures found on CIFAR-10 and
CIFAR-100 can be transferred to ImageNet-16-120
with good performance.
In LoNAS, two computational constraints
are designed to limit the number
ofnetwork architecture parameters in the
search space; these constraints compress
the search space and reduce the diversity
to some extent. Future work will explore
other methods to balance the trade-off
between multiple objectives in larger
search spaces. In addition, the NTK
should be improved since the performance
oftheNTK conditions fluctuates.
Acknowledgment
This work was supported in part by the
National Natural Science Foundation
of China under Grants 62073155,
62002137, 62106088, and 62206113;
in part by Jiangsu University, China
through Blue Project; and in part by
Guangdong Provincial Key Laboratory
under Grant 2020B121201001.
References
[1] K. Simonyan and A. Zisserman, " Very deep
convolutional networks for large-scale image recognition, "
in Proc. Int. Conf. Learn. Representations,
2015.
[2] K. He, X. Zhang, S. Ren, and J. Sun, " Deep
residual learning for image recognition, " in Proc.
IEEE Conf. Comput. Vis. Pattern Recognit., 2016,
pp. 770-778.
[3] C. Szegedy et al., " Going deeper with convolutions, "
in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
2015, pp. 1-9.
[4] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q.
Weinberger, " Densely connected convolutional networks, "
in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
2017, pp. 4700-4708.
[5] B. Baker, O. Gupta, N. Naik, and R. Raskar,
" Designing neural network architectures using reinforcement
learning, " 2016, arXiv:1611.02167.
[6] A. Brock, T. Lim, J. Ritchie, and N. Weston,
" SMASH: One-shot model architecture search
through hypernetworks, " in Proc. Int. Conf. Learn.
Representations, 2018.
[7] E. Real, A. Aggarwal, Y. Huang, and Q. V. Le,
" Regularized evolution for image classifier architecture
search, " in Proc. AAAI Conf. Artif. Intell., 2019,
vol. 33, pp. 4780-4789.
[8] Y. Sun, B. Xue, M. Zhang, and G. Yen,
" Completely automated CNN architecture design
based on blocks, " IEEE Trans. Neural Netw. Learn.
Syst., vol. 31, no. 4, pp. 1242-1254, Apr. 2020.
[9] Y. Sun, B. Xue, M. Zhang, G. G. Yen, and J. Lv,
" Automatically designing CNN architectures using
the genetic algorithm for image classification, " IEEE
Trans. Cybern., vol. 50, no. 9, pp. 3840-3854, Sep.
2020.
[10] Z. Sun, M. Lin, X. Sun, Z. Tan, H. Li, and R.
Jin, " MAE-Det: Revisiting maximum entropy principle
in zero-shot NAS for efficient object detection, "
in Proc. Int. Conf. Mach. Learn., 2022, pp. 20810-
20826.
[11] S. Liu et al., " EVSRNet: Efficient video superresolution
with neural architecture search, " in Proc.
IEEE/CVF Conf. Comput. Vis. Pattern Recognit.,
2021, pp. 2480-2485.
[12] S. Xu and H. Quan, " ECT-NAS: Searching efficient
CNN-transformers architecture for medical
image segmentation, " in Proc. IEEE Int. Conf. Bioinf.
Biomed., 2021, pp. 1601-1604.
[13] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le,
" Learning transferable architectures for scalable image
recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., 2018, pp. 8697-8710.
[14] E. Real et al., " Large-scale evolution of image
classifiers, " in Proc. Int. Conf. Mach. Learn., 2017,
pp. 2902-2911.
[15] A. G. Howard et al., " MobileNets: Efficient convolutional
neural networks for mobile vision applications, "
2017, arXiv:1704.04861.
[16] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov,
and L.-C. Chen, " MobileNetv2: Inverted residuals
and linear bottlenecks, " in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit., 2018, pp. 4510-4520.
[17] Z. Lu et al., " NSGA-Net: Neural architecture
search using multi-objective genetic algorithm, " in
Proc. Genet. Evol. Comput. Conf., 2019, pp. 419-427.
[18] T. Elsken, J. H. Metzen, and F. Hutter, " Efficient
multi-objective neural architecture search via
lamarckian evolution, " in Proc. Int. Conf. Learn. Representations,
2018.
[19] T. Back, Evolutionary Algorithms in Theory and Practice:
Evolution Strategies, Evolutionary Programming, Genetic
Algorithms. London, U.K.: Oxford Univ. Press, 1996.
[20] L. Xie and A. Yuille, " Genetic CNN, " in Proc.
IEEE Int. Conf. Comput. Vis., 2017, pp. 1379-1388.
92 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2023
[21] O. Russakovsky et al., " ImageNet large scale
visual recognition challenge, " Int. J. Comput. Vis.,
vol. 115, no. 3, pp. 211-252, 2015.
[22] A. Krizhevsky, I. Sutskever, and G. E. Hinton,
" ImageNet classification with deep convolutional
neural networks, " in Proc. Adv. Neural Inf. Process.
Syst., 2012, vol. 25, pp. 1097-1105.
[23] T. Zhang, C. Lei, Z. Zhang, X.-B. Meng, and
C. P. Chen, " AS-NAS: Adaptive scalable neural
architecture search with reinforced evolutionary algorithm
for deep learning, " IEEE Trans. Evol. Comput.,
vol. 25, no. 5, pp. 830-841, Oct. 2021.
[24] H. Cai, L. Zhu, and S. Han, " ProxylessNAS: Direct
neural architecture search on target task and hardware, "
in Proc. Int. Conf. Learn. Representations, 2018.
[25] B. Wu et al., " FBNet: Hardware-aware efficient
convnet design via differentiable neural architecture
search, " in Proc. IEEE/CVF Conf. Comput. Vis. Pattern
Recognit., 2019, pp. 10734-10742.
[26] Z. Lu, K. Deb, E.Goodman,W.Banzhaf, and
V. N. Boddeti, " NSGANetv2: Evolutionary multiobjective
surrogate-assisted neural architecture search, "
in Proc.Eur.Conf. Comput.Vis., 2020, pp. 35-51.
[27] X. Dong and Y. Yang, " Searching for a robust
neural architecture in four GPU hours, " in Proc.
IEEE/CVF Conf. Comput. Vis. Pattern Recognit.,
2019, pp. 1761-1770.
[28] H. Liu, K. Simonyan, and Y. Yang, " Darts: Differentiable
architecture search, " in Proc. Int. Conf.
Learn. Representations, 2018.
[29] C. Liu et al., " Auto-deeplab: Hierarchical neural
architecture search for semantic image segmentation, "
in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.,
2019, pp. 82-92.
[30] H. Pham, M. Guan, B. Zoph, Q. Le, and J.
Dean, " Efficient neural architecture search via parameters
sharing, " in Proc. Int. Conf. Mach. Learn., 2018,
pp. 4095-4104.
[31] K. Yu, C. Sciuto, M. Jaggi, C. Musat, and M.
Salzmann, " Evaluating the search phase of neural
architecture search, " in Proc. Int. Conf. Learn. Representations,
2020.
[32] S. Xie, A. Kirillov, R. Girshick, and K. He,
" Exploring randomly wired neural networks for
image recognition, " in Proc. IEEE/CVF Int. Conf.
Comput. Vis., 2019, pp. 1284-1293.
[33] C. Liu et al., " Progressive neural architecture search, "
in Proc.Eur.Conf.Comput. Vis., 2018, pp. 19-34.
[34] H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han,
" Once-for-all: Train one network and specialize it for
efficient deployment, " in Proc. Int. Conf. Learn. Representations,
2019.
[35] R. Luo, X. Tan, R. Wang, T. Qin, E. Chen, and
T.-Y. Liu, " Semi-supervised neural architecture
search, " in Proc. Adv. Neural Inf. Process. Syst.,2020,
vol. 33, pp. 10547-10557.
[36] J. Mellor, J. Turner, A. Storkey, and E. J.
Crowley, " Neural architecture search without
training, " in Proc.Int.Conf.Mach.Learn., 2021,
pp. 7588-7598.
[37] W. Chen, X. Gong, and Z. Wang, " Neural
architecture search on imagenet in four GPU hours: A
theoretically inspired perspective, " in Proc. Int. Conf.
Learn. Representations, 2020.
[38] A. Jacot, F. Gabriel, and C. Hongler, " Neural
tangent kernel: Convergence and generalization in
neural networks, " in Proc. 32nd Int. Conf. Neural Inf.
Process. Syst., 2018, pp. 8580-8589.
[39] B. Hanin and M. Nica, " Finite depth and width
corrections to the neural tangent kernel, " in Proc. Int.
Conf. Learn. Representations, 2019.
[40] Y. Xu et al., " PC-darts: Partial channel connections
for memory-efficient architecture search, " in
Proc. Int. Conf. Learn. Representations, 2019.
[41] S.-Y. Huang and W.-T. Chu, " Searching by
generating: Flexible and efficient one-shot NAS with
architecture generator, " in Proc. IEEE/CVF Conf.
Comput. Vis. Pattern Recognit., 2021, pp. 983-992.
[42] A. Klos, M. Rosenbaum, and W. Schiffmann,
" Neural architecture search based on genetic algorithm
and deployed in a bare-metal kubernetes cluster, " Int.J.
Netw. Comput.,vol. 12,no. 1, pp.164-187, 2022.
IEEE Computational Intelligence Magazine - May 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2023
