IEEE Computational Intelligence Magazine - August 2021 - 8

Guest
Yanan Sun
Editorial
Sichuan University, CHINA
Mengjie Zhang
Victoria University of Wellington,
NEW ZEALAND
Gary G. Yen
Oklahoma State University, USA
Evolutionary Neural Architecture Search and Applications
D
eep neural networks (DNNs) have
shown significantly promising performance
in addressing real-world
problems, such as image recognition, natural
language processing and self-driving.
The achievements of DNNs owe largely
to their deep architectures. However,
designing an optimal deep architecture for
a particular problem requires rich domain
knowledge on both the investigated data
and the neural network domains, which is
not necessarily held by the end-users.
Neural architecture search (NAS), as an
emerging technique to automatically
design the optimal deep architectures
without requiring such expertise, is drawing
increasing attention from industry and
academia. However, NAS is theoretically a
non-convex and non-differentiable optimization
problem, and existing methods
are incapable of well addressing it. Evolutionary
computation approaches, particularly
genetic algorithms, particle swarm
optimization and genetic programming,
have shown superiority in addressing realworld
problems due largely to their powerful
abilities in searching for global
optima, dealing with non-convex/nondifferentiable
problems, and requiring no
rich domain knowledge. In this regard,
deep neural architecture designed by evolutionary
computation approaches, so
called evolutionary neural architecture
search (ENAS), have attracted the interest
of many researchers.
Digital Object Identifier 10.1109/MCI.2021.3084391
Date of current version: 15 July 2021
This special issue has brought together
researchers to report state-of-the-art contributions
on the latest research and development,
up-to-date issues, challenges, and
applications in the field of ENAS. Following
a rigorous peer review process, five
papers have been accepted for publication
in this special issue.
The first paper included in the special
issue is entitled " Evolutionary MultiObjective
Model Compression for Deep
Neural Networks " authored by Z. Wang et
al., which aims at accelerating the inference
speed of DNNs by optimizing the model
size and accuracy simultaneously with evolutionary
algorithms. The architecture population
evolution was employed to explore
and exploit the network space of pruning
and quantization. In addition, a two-stage
co-optimizing strategy of pruning and
quantization was proposed to significantly
reduce time cost during the architecture
search process. To further lower energy
consumption and reduce the model size,
various dataflow designs and parameter
coding schemes were also considered in the
optimization process. Unlike most related
research solely focusing on reducing the
model size while maintaining the model
accuracy, the work in this paper can achieve
a trade-off between different model sizes
and model accuracies, which meets the
requirements for most edge devices in realworld
scenarios. The experimental results
demonstrated that the proposed algorithm
can obtain a broad range of compact
DNNs for diverse memory usage and
energy consumption requirements.
8 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2021
The second paper, titled " Fast and
Unsupervised Neural Architecture Evolution
for Visual Representation Learning "
by S. Xue et al., proposes the FaUNAE
algorithm to improve the unsupervised
visual representation learning ability
against the supervised peer competitors.
Specifically, FaUNAE employed the evolutionary
algorithm to search for promising
neural architectures from an existing
architecture designed by experts or existing
NAS algorithms that focus on the
transferability from a small dataset to a
larger dataset. To reduce the search cost
and enhance the search efficiency, the
prior knowledge and the inferior, as well
as least promising operations, were used
during the evolutionary process. In addition,
the contrast-loss function was utilized
as the evaluation metric in a
student-teacher framework to achieve the
self-supervised evolution. FaUNAE was
evaluated on four widely used large-scale
benchmark datasets. The results demonstrated
the effectiveness of FaUNAE
upon various downstream applications,
including object recognition, object
detection, and instance segmentation.
The third paper entitled " Self-Supervised
Representation Learning for Evolutionary
Neural Architecture Search " by
C. Wei et al. proposes a novel performance
predictor, aiming to enhance the efficiency
of ENAS algorithms. Specifically, the
ENAS algorithms are often computationally
expensive in practice because hundreds
of DNNs needed to be trained
during the search process. Performance
IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
