IEEE Computational Intelligence Magazine - May 2023 - 81

formed by the remaining individuals
enters into the next round. The evolution
process continues until the predefined
maximum number of evolution
rounds is reached. Finally, the best network
architecture is decoded from the
individual in the final population with
the best fitness. The corresponding
flowchart is shown in Fig. 1.
FIGURE 1. The flowchart of the LoNAS algorithm, where i is the evolution round, Max gen is the
maximum number of evolution rounds, and G1 and G2 are the two separation points in the threestage
evolution process.
Each individual represents a network
architecture encoded based on the variable-architecture
encoding strategy. Various
network architectures can be
constructed by searching for different
parameter combinations, which helps to
expand the search space. The condition
number of the NTK (KN) is set as the
training-free proxy to evaluate individuals.
The evaluation time is significantly
reduced since complete network training
is avoided (Line 2). The iterative evolution
process is divided into three stages
according to the different environmental
selection criteria. The first and third
stages use KN as the criterion to implement
environmental selection, and the
second stage leverages the individual lifespan
Lifespan as the criterion. Several
individuals in the population are selected
as parents based on tournament selection
(Line 5). The offspring are generated
from the parents through mutation operators
(Line 6). After the offspring are constructed,
they are evaluated and added to
the population once constructed. The
corresponding criterion is used for environmental
selection according to the
stage of the current evolution round.
Some individuals are eliminated (Line 8
and Line 10) to ensure that the number
of surviving individuals in the population
is consistent with that in the initial
population. Then, a new population
B. Expanded Search Space Based on
the Reg Block
The network architectures in LoNAS
are constructed by a variable-architecture
encoding strategy based on a
novel network block called the Reg
Block. A well-designed search space
containing numerous potential network
architectures is crucial for NAS.
Therefore, five dimensions are
included to construct the LoNAS
search space: the length of the network
architecture, the number of
group convolutions, the width of each
group convolution, the stride of the
pooling layers, and the SENet module.
Compared with the search spaces
with fewer dimensions in [8], [9],
[46], the search space is expanded in
LoNAS since different network architectures
can be constructed in more
flexible and fine ways. More potential
network architectures can be searched,
which leads to better solutions. An
overview of the search space and the
encoding strategy is shown in Fig. 2.
FIGURE 2. LoNAS search space and the encoding strategy. (a): The macro skeleton of the network architecture. (b): The topology of the Reg Block.
(c): The considered options for the network architecture parameters. (d):A flexible encoding strategy based on the Reg Block.
MAY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 81

IEEE Computational Intelligence Magazine - May 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2023