IEEE Computational Intelligence Magazine - May 2023 - 28

TABLE II Summary of HV and IGD+ comparisons between
CMOQLMT and CMODQLMT on benchmark CMOPs.
HV þ= = ¼
3/0/7
IGD+ þ= = ¼
Benchmark CMOQLMT vs CMODQLMT CMOQLMT vs CMODQLMT
DTLZ
MW
1/0/9
LIR-CMOP
DAS-CMOP
1/0/13
4/5/5
0/2/7
1/1/12
4/6/4
0/0/9
TABLE III Average rankings by Friedman test of CMODQLMT
and other methods on run time.
ALGORITHM
BiCo
CCMO
CMOEA-MS
DSPCMDE
URCMO
c-DPEA
MTCMO
PPS
CCEA
DC-NSGA-III
ICMA
CMODQLMT
RANKING
3.63830
7.3830
4.2660
2.4894
8.3511
9.2766
11.3191
5.8830
7.6915
1.0000
5.0426
11.6596
❏ vs CMOQLMTallAuxiliary: Adopting all auxiliary tasks
would divide the evaluations equally and result in a lack of
selection for useful auxiliary tasks.
❏ vs CMOQLMTnoStage2: Without the evolving stage, the
evaluations were not concentrated on suitable tasks learned
by QL.
D. ParameterAnalysis
The proposed methods contain two parameters, one is l used
to control the length of the learning stage, and the other is
the greedy factor " . This paper sets l ¼f0:3; 0:5; 0:7g and
" ¼f0:8; 0:9; 1:0g. Among these values, l ¼ 0:5 and " ¼ 0:9
are used in CMOQLMT. The HV and IGD+ results are
reported in Table S-VI in the supplementary file. For Type-III
problems, a smaller l (i.e., shorter learning stage) performed
better because T3 can be easily determined as the best auxiliary
task for this type ofproblem. In contrast, a larger l resulted in a
shorter evolving stage, degenerating the performance due to
the degenerated exploitation. A smaller " provided the random
selection probability, slightly degenerating the results. When "
is set to one, the exploration degenerated because the algorithm
only selected the auxiliary task based on the learned
Q-Table. Therefore, except for several instances ofDTLZ, the
performance degenerated when " is set to one.
It should be noted that except for the extra introduced
parameters, the parameters of the primary tools (evolutionary
28 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2023
FIGURE 9 Final CPFs and Q-Tables obtained by CMOQLMTon DC3DTLZ3,
MW13 and LIR-CMOP7 with the median IGD+ values among 30
runs.
operators and neural network) all remain the same as common
settings in the corresponding fields and are not fine-tuned to
avoid labor-intensive work because this work focuses on
adopting RL techniques in the EMT-based auxiliary task
selection issue.
E. CMODQLMTResults
CMODQLMT is compared with CMOQLMT to verify the
effectiveness and efficiency of the proposed DQL-based
method. The statistical results of HV and IGD+ on the five
selected benchmarks are reported in Tables S-VII and S-VIII,
respectively. The results are summarized in Table II. Specifically,
CMODQLMT obtained competitive results compared
to CMOQLMT. CMOQLMT provided slightly better results
on DTLZ and MW benchmarks, and some instances of LIRCMOP.
However, CMODQLMT obtained slightly worse
results on DAS-CMOP and some LIR-CMOP problems.
Figures S6 to S9 present the CPFs obtained by CMODQLMT
on all instances. Similar to CMOQLMT, CMODQLMT
could also approximate the true CPF and obtain good distribution
on each instance, indicating that the effectiveness of the
proposed DQL-based evolution. For the relatively simple
IEEE Computational Intelligence Magazine - May 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2023
