IEEE Computational Intelligence Magazine - August 2019 - 69

B. Results and Analysis

Fig. 8 shows the convergence profiles of
GD values obtained by SPEA2, IBEA,
NSGA-III, BCE-IBEA, and SMS-EMOA
on DTLZ1 and DTLZ2, averaged over 30
runs. It can be seen that all the MOEAs
exhibit a good convergence performance,
hence the difference between their performance mainly lies in diversity. The nondominated solution sets obtained in one
run of the five MOEAs on 3-objective
DTLZ1-DTLZ7, CWDV, and 8-objective ML-DMP are plotted in Fig. 1 in
the Supplementary Materials. It can be
found from the figure that SPEA2,
NSGA-III, and BCE-IBEA exhibit a
good diversity performance on DTLZ1-
DTLZ4, SPEA2, IBEA, and BCE-IBEA
can obtain a population with good
diversity on DTLZ5-DTLZ7 and

101

Three-Objective DTLZ1

5

10-1

10-4
30

GD

GD

Specifically, the fitness scaling factor in
IBEA and BCE-IBEA is set to 0.0001
for DTLZ1 and DTLZ3, and 0.005 for
the remaining MOPs. The population
size N of all the compared MOEAs is
set to 100, 105, and 156 for the MOPs
with two, three, and eight objectives,
respectively. The maximum number of
generations is set to 300, which is
enough for the MOEAs to converge. As
for the genetic operators, all the compared MOEAs adopt simulated binary
crossover (SBX) [39] and polynomial
mutation [40], where the probabilities of
crossover and mutation are set to 1 and
1/D (D denotes the number of decision
variables), respectively, and the distribution index of both SBX and polynomial
mutation is set to 20.

120
210
Generations
(a)
SPEA2

IBEA

300

NSGA-III

×10-3 Three-Objective DTLZ2

3

0

30

120
210
Generations
(b)

BCE-IBEA

300

SMS-EMOA

FIGURE 8 Convergence profiles of GD values obtained by SPEA2, IBEA, NSGA-III, BCE-IBEA, and
SMS-EMOA on 3-objective DTLZ1 and DTLZ2.

CWDV, and the diversity performance
of IBEA is significantly better than the
others on ML-DMP.
To quantitatively compare the diversity
performance of the compared MOEAs,
the obtained non-dominated solution sets
in objective space are assessed by seven
performance metrics, namely, Spacing,
CL n, PD, IGD, T p, HV, and the proposed
CPF. The parameters in these metrics are
set to the same as introduced in Section
II-B. Besides, for IGD, T p, and CPF,
roughly 10,000 reference points on the
Pareto front of each MOP are sampled
using the methods in [41]; for HV, the reference point is set to (1.1, f, 1.1) and the
objective values are normalized by the
nadir point of the Pareto front. Besides, the
Wilcoxon rank sum test with a significance level of 0.05 is also adopted to analyze the result, where '+', '-' and '≈'
indicate that the result is significantly bet-

ter, significantly worse, and statistically similar to that obtained by the result in the last
column (i.e., SMS-EMOA), respectively.
The metric values of Spacing, CL n,
PD, IGD, T p, and HV are listed in Table I
in the Supplementary Materials. Note
that the T p values are totally the same to
the IGD values since all the compared
MOEAs have good convergence performance, where T p is equivalent to IGD in
this case. It can be seen from the table that
each metric indicates a totally different
observation. In fact, according to the
results plotted in Fig. 1 in the Supplementary Materials, all the metrics shown
in Table I in the Supplementary Materials
are counter-intuitive to some extent. To
be specific, SPEA2 obtains the best Spacing value on DTLZ7 and CWDV, but the
solution set obtained by SPEA2 is less
uniform than that obtained by BCEIBEA. NSGA-III and SMS-EMOA

TABLE III CPF values obtained by SPEA2, IBEA, NSGA-III, BCE-IBEA, and SMS-EMOA on 3-objective DTLZ1-DTLZ7, CWDV, and
8-objective ML-DMP, averaged over 30 Runs. The best result in each row is highlighted.
PROBLEM

SPEA2

IBEA

NSGA-III

BCE-IBEA

SMS-EMOA

DTLZ1

6.5305e-1 (2.25e-2) +

5.6315e-1 (3.35e-2) +

6.9161e-1 (2.15e-2) +

6.4244e-1 (2.42e-2) +

2.2337e-1 (5.48e-2)

DTLZ2

7.0532e-1 (2.60e-2) +

4.0179e-1 (3.18e-2) +

6.9970e-1 (4.21e-3) +

6.8451e-1 (2.76e-2) +

3.1461e-1 (3.70e-2)

DTLZ3

7.3148e-1 (2.33e-2) +

3.7824e-1 (3.30e-2) +

6.7190e-1 (3.82e-2) +

5.8869e-1 (5.70e-2) +

2.0930e-1 (4.17e-2)

DTLZ4

7.1266e-1 (3.30e-2) +

3.9213e-1 (9.10e-2) +

7.0133e-1 (4.57e-3) +

6.6854e-1 (2.40e-2) +

2.5240e-1 (1.23e-1)

DTLZ5

9.4123e-1 (7.27e-3) +

9.0703e-1 (2.72e-2) +

5.3304e-1 (4.41e-2) −

9.1546e-1 (7.51e-3) +

6.9035e-1 (5.78e-2)

DTLZ6

9.2594e-1 (1.02e-2) +

5.8801e-1 (3.75e-2) ≈

2.8130e-1 (3.68e-2) −

9.1076e-1 (4.99e-3) +

5.5227e-1 (7.75e-2)

DTLZ7

6.9931e-1 (3.09e-2) +

5.4629e-1 (6.69e-2) +

3.6519e-1 (3.69e-2) ≈

7.9022e-1 (1.31e-1) +

3.5745e-1 (4.53e-2)

CWDV

8.0441e-2 (1.27e-2) +

1.1424e-1 (1.34e-2) +

4.1616e-2 (1.22e-2) −

1.6833e-1 (1.05e-2) +

5.8490e-2 (1.30e-2)

ML-DMP

8.4886e-1 (9.68e-2) +

8.6421e-1 (7.62e-3) +

6.1208e-1 (5.09e-2) −

6.9916e-1 (1.15e-1) ≈

7.0131e-1 (6.58e-2)

AUGUST 2019 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

69
IEEE Computational Intelligence Magazine - August 2019

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2019
