IEEE Robotics & Automation Magazine - June 2023 - 18

Since different training orders affect
the accuracy of tasks, in this article we
average the ACC and BWT under different
training orders to obtain the comprehensive
accuracy (CACC) and comprehensive
BWT (CBWT), which are defined as:
CACC = 1 / ACC()n
N n=1
N
(8)
CBWT= 1 / BWT()n
N n=1
N
(9)
where N denotes the number of different
orders. In our experiments, we have tried
three different task orders, and thus
N 3= is in our experiment.
Finally, the average error (AERR) of
the ACC and the CACC under different
training orders is used to evaluate the
effect of the training order on the model
performance.
AERR
==
N
n
RESULT ANALYSIS
COMPARISON RESULTS
Figure 7 demonstrates the forgetting curves of different algorithms
under task orders of (a), (b), and (c), respectively. It can
be seen that the proposed RGO method obviously outperforms
the EWC, LWF, GPM, LWM, and SGD methods. The
EWC, LWF, LWM, and SGD methods reduce the classification
accuracy on previous tasks significantly and cannot
maintain the stability of the model. In addition, as the LWF
and EWC methods take the loss function of old tasks into
consideration, and the GPM method learns new tasks based
on the gradient subspaces of old tasks, the plasticity of the
model for new tasks is thus reduced with these methods. As a
result, a decrease in the accuracy for the new task happens
TABLE 1. The number of samples of six industrial
defect data sets.
NUMBER OF TRAIN
SET SAMPLES/PIECE
NORMAL
SAMPLE
TPD
TPM
304
70
TCW 248
TSW
CCC
CBD
300
400
200
DEFECT
SAMPLE
51
70
42
50
400
200
NUMBER OF TEST
SET SAMPLES/PIECE
NORMAL
SAMPLE
49
50
58
50
200
100
18 IEEE ROBOTICS & AUTOMATION MAGAZINE JUNE 2023
DEFECT
SAMPLE
51
50
42
50
200
100
1
1 / ACC CACC .
()n
N
(10)
FORGETTING PROBLEM
AND REALIZE THE
CONTINUOUS
AND EFFECTIVE
CLASSIFICATION OF
VARIOUS INDUSTRIAL
DEFECTS.
„
"
THE EXPERIMENTAL
RESULTS DEMONSTRATE
THAT THE PROPOSED
FRAMEWORK CAN
GREATLY ALLEVIATE
THE CATASTROPHIC
compared with the MTL and RGO methods.
On the other hand, the proposed
RGO method restricts gradient updating,
and at the same time, the virtual FEL is
introduced to reduce the interference
between different tasks. Therefore, it can
maintain the classification accuracy of
previous tasks and achieve satisfying
performance for new tasks. We can see
that the classification accuracy of the
current task with the RGO method is
close to the upper-bound MTL method.
Also, the RGO method shows stronger
stability than the EWC, LWF, GPM,
LWM, and SGD methods while maintaining
its plasticity.
The ACC of six industrial defect data
sets in different orders are shown in Figure
8. Due to the large difference in characteristics
between different tasks, we can
observe a serious forgetting phenomenon
under different task orders with EWC,
LWF, LWM, and SGD methods, and the
trained models are not able to adapt to various industrial defects
in production lines effectively. The GPM method performs better
than the EWC, LWF, LWM, and SGD methods, while it does
not work as well as the proposed method due to its low plasticity.
Basically, the ACC of the RGO method under different
task orders is significantly higher than that of the EWC, LWF,
LWM, GPM, and SGD methods, and it is quite close to the ACC
of the upper-bound MTL method. The experimental results
demonstrate that the proposed framework can greatly alleviate
the catastrophic forgetting problem and realize the continuous
and effective classification of various industrial defects.
We also list results of CACC, CBWT, and AERR of different
methods (Table 2). It can be seen that the comprehensive
accuracy of RGO is significantly higher than that of the
EWC, LWF, LWM, GPM, and SGD methods, which is only
slightly worse than the upper-bound MTL method. EWC does
not significantly alleviate the forgetting problem, as expected,
and LWF even exacerbates catastrophic forgetting. Although
GPM can alleviate catastrophic forgetting problem well, it still
performs worse than the proposed RGO method. In addition,
the RGO method is obviously less affected by task orders compared
with the EWC, LWF, LWM, and GPM methods, with an
average error of .6633%.
We further dig into the results of the SGD method, which
does not consider the catastrophic forgetting problem at all in
details (Table 3). We find that the trained model with SGD is
actually faced with a serious catastrophic forgetting problem.
In task order (a), all samples in task 1, task 2, and task 4 are
identified as normal samples, and all samples of task 3 are
identified as defect samples. In task order (b), most samples
in task 1 are identified as defect samples, all samples in task 3
are identified as defect samples, and all samples in task 4 and
task 5 are identified as normal samples. In task order (c), all
IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
