IEEE Computational Intelligence Magazine - May 2023 - 83

NTK can be used to characterize the trainability of each
network architecture and therefore is designed to
evaluate the performance of individuals in the
population that can significantly save search time and
computational resources.
most informative components of the features,
thereby improving the representational
capacity of the network
architecture.
3)Variable-Architecture Encoding
Strategy
A variable-architecture encoding strategy
based on the Reg Block is designed
to construct a large search space with
numerous network architectures, as
shown in Fig. 2(c) and (d). Searching
different parameter combinations allows
various network architectures to be
flexibly constructed. First, the value of
the unit num parameter is randomly
selected to determine the initial number
ofReg Units. Then, in each Reg Unit,
block num Reg Blocks are randomly
generated.
As depicted in Fig. 2(b), the input of
each Reg Block comes from the output
of the previous Reg Block. The input
feature maps are divided into group
branches.Eachbranchhas g channel
channels, which are determined by
in channel=group. The width of the bottleneck
ofeach branch is set as the width.
The stride ofthe pooling layer is denoted
asf.f is set to 2 when the feature map is
downsampled; otherwise,f is set to 1. At
the end of each Reg Block, an SENet
module is added with a 50% probability,
which improves the representational ability
of the network without introducing
too many parameters. This dimensionality
is denoted by the parameter
hasSENet. The value ofhasSENet is set to
either 1 or 0, representing whether an
SENet module is in the Reg Block.
When generating a Reg Block, the
parameters group and width are randomly
selected from two specified lists. The
parameter f is randomly selected from
{1, 2}. When the number of existing
pooling layers performing the halving
meets the preset constraint,f is set to 1.
These architectural parameters are automatically
searched and encoded as a digital
string.
By including the group convolution
and SENet module, more dimensions for
constructing network architectures are
added, effectively expanding the search
space and allowing more networks with
better performance to be obtained.
4)Computational Restriction
The length of the network architecture
has an important effect on the test
accuracy [1]. Longer network architectures
generally obtain better test accuracy
but require more parameters,
which makes these network architectures
more challenging to calculate.
Through careful structure design, some
short network architectures can achieve
good classification performance [2], [4],
and determining these architectures is
the goal of the algorithm proposed in
this paper. Although the lengths of the
network architectures in the LoNAS
search space can vary according to the
encoding strategy, the lengths are
restricted to ensure that the number of
parameters of each network architecture
in the search space is limited. Network
architectures with too many or
too few parameters can be avoided
since they are not the target, thereby
improving the search efficiency. The
number of convolutional layers determines
the length of the network architecture:
Length
¼ 1 þ
UXmax
i¼Umin
where unit num is limited to the range
½Umin; Umax. Ublock
i
represents the number
of Reg Blocks in the ith Reg Unit,
which corresponds to the parameter
block num. block num is limited to the
range ½blockmin; blockmax. The number
3 Ublock
i
(1)
of Reg Blocks in each Reg Unit must
be multiplied by a factor of 3 since
each Reg Block contains three convolutional
layers. The total length of the
network architecture is determined by
adding a convolutional layer in the
Conv Unit.
The number of convolutional kernels
also affects the number of network
parameters. When the convolutional
kernels are the same size, more convolutional
kernels improve the feature
extraction ability, leading to better classification
performance for the network
architecture while requiring more
parameters. Therefore, the number of
convolutional kernels should also be
limited to avoid requiring too many
parameters. The number of convolutional
kernels reflects the number of
output feature maps in the intermediate
layer, which is determined by the product
of the group and width parameters.
The proposed encoding strategy implements
two rules to restrict the number
of feature maps. The first rule is that
larger values have lower selection probabilities
when randomly selecting the
group and width parameters. The other
rule is that maximum and minimum
values are set for the product of group
and width.Ifgroup width is larger than
the maximum value or less than the
minimum value, both parameters need
to be randomly selected again until the
product meets the limit. Algorithm 2
shows the application of these rules in
the construction of the network architecture.
Under these rules, the overall
number of network architecture parameters
in the search space is limited,
which guarantees better search performance
to discover highly accurate network
architectures with few parameters.
Compared to the coding strategies
in [9], [26], [46], there are more parameter
dimensions included in the proposed
encoding strategy. As a result, finer
blocks can be generated, and more various
network architectures can be discovered.
In addition, network architectures
with too few or too many parameters
that do not meet the parameter requirements
are not generated by adding
computational restrictions, resulting in a
MAY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 83
IEEE Computational Intelligence Magazine - May 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2023
