IEEE Robotics & Automation Magazine - June 2023 - 13

machine (SVM) classifier is used to realize
solder joint defect classification. And
then, with the great success of deep learning
techniques, the traditional machine
learning methods are gradually replaced
by deep learning ones in the defect
inspection task. Masci et al. [4] propose a
max-pooling convolutional neural network
(CNN) approach for the steel defect
classification task, and it achieves a two
times greater performance than the SVM
methods. With the fast development of
deep learning in computer vision, the
defect detection task can also leverage
results from other computer vision tasks,
such as image segmentation [5].
In some situations, the components
"
IT IS NECESSARY TO
DEVELOP A LIFELONG
LEARNING DEFECT
RECOGNITION
FRAMEWORK TO
CONTINUOUSLY UPDATE
THE OLD MODEL TO
ADAPT TO NEW
INDUSTRIAL SCENARIOS.
„
to be inspected could have overlapping
defects. Therefore, it is reasonable to employ a multitarget
multiclass defects classification method to tackle the task. In
[6], the feature map groups of different types of defects are
extracted by different convolutional kernels, and then each
feature map is fed into a detection network regressing different
defects. Metalearning approaches [7] have also tried to find
suitable CNN architectures for the multiclass multitarget task
and have yielded even better performance than existing CNN
architectures. Additionally, the attention mechanism could be
beneficial for this challenging task. Bhattacharya et al. [8] propose
a deep multideformation aware attention learning architecture,
which is able to extract crucial feature from multiple
scales and localize the specific multitarget multiclass defects.
LIFELONG LEARNING
The catastrophic forgetting problem is still the greatest challenge
in lifelong learning and a detailed introduction of existing
approaches is presented in [2], which divides them into
three categories, namely the replay
method, parameter isolation method,
and regularization-based method.
The replay method is the most intuitive
way to solve the catastrophic forgetting
problem. It uses old task samples or
pseudosamples generated by generative
models to retrain the model when new
task samples arrive, while extra memory
and computing resources are required
for the data storage and computing. In
the incremental classifier and representation
learning (iCaRL) method [9], only
a small number of exemplars per class
are stored and it proposes to learn the
strong classifier and data representation
simultaneously to avoid catastrophic
forgetting. Gradient episodic memory
[10] proposes an episodic memory that
stores a part of the data and projects
the estimated gradient to the closest gradient
satisfying the old task constraints.
Besides past samples, Shin et al. [11] propose
a deep generative replay framework
in which a generative model is trained to
mimic past data and the generated data are
used to retrain the network when a new
task arrives.
In the parameter isolation method,
for each task a group of parameters is
assigned and the size of the model could
be changed. A progressive network is proposed
in [12], where a new layer is added
after the previous layer when a new task
comes and previous layers are frozen.
To keep the architecture of the model
unchanged, some methods try to mask out
parts of the model when training the new
task [13]. However, the above methods do not consider which
task model is deployed when a new sample comes in the test.
To this end, Aljundi et al. [14] propose an " expert gate, " which
includes a set of gating autoencoders and can automatically forward
the test sample to the relevant expert, while the size of the
model increases linearly with the increase of the tasks.
Another class of regularization-based method is also usually
employed for lifelong learning, which introduces a regularization
term to consolidate previous knowledge in the loss
function, avoiding the extra memory requirement in the replay
method. In [15], outputs from the old model are used to preserve
the performance on old tasks and act as a regularizer to
improve the performance on the new task. The elastic weight
consolidation (EWC) method constrains changes of important
parameters in previous tasks [16]. Another popular approach is
the learning without memorizing (LWM) method [17], in which
an information-preserving penalty is used to penalize changes
in classifiers' attention maps to retain information of old tasks.
Data Stream
Different Production Lines in Industrial Scenarios
FIGURE 2. With data from new scenarios coming, the performance of the model on the old
tasks seriously decreases and thus catastrophic forgetting occurs.
JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
13
IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
