IEEE Computational Intelligence Magazine - May 2021 - 28

values given by both xNES and SGD are around 1e-3 to 1e-5
and could not be further reduced. This is mainly caused by the
neural networks' inability to emulate the right solution at steep
gradient region. Transferred from other diffusion coefficients,
tNES achieves the best optimized loss among all 3 techniques.

lide. After the collision, they split and regain their magnitudes.
Figure 8c shows such interaction for o = 0.001 using the solution optimized by tNES (with o = 0.0008 as source prior).
Figure  8f compares the mean absolute residual (aggregated
residual from differential equation and initial condition)
between tNES and SGD solutions for the same case, based on
20 independent optimization runs. The results suggest a more
accurate solution from tNES across the computational domain.
Figure  8g shows the distribution of the optimized loss based
on multiple tNES, xNES and SGD optimization runs for dispersive coefficient o = 0.0005, 0.0010, 0.0015. Transferred
from other diffusion coefficients, tNES improves the optimized
loss significantly in comparison to both SGD and xNES.

3) Korteweg-de Vries (KdV) Equation
The KdV equation [56] is used in physics and engineering to
model the weakly nonlinear long waves (for example, waves on
shallow water surfaces). It has several variants, and we consider
the KdV equation in the form:
u t + u $ u x = o $ u xxx, x e 60, 1.5@, t e 60, 2@ (19)

	

D. Study of Mixing Coefficients in tNES

which consists of a dispersive coefficient o. The solution of
(19) describes the height of the wave at position x and
time t. We consider the following initial condition [57]:
u ^x, 0 h = 3c 1 sech 2 a 1 ^x - x 1h + 3c 2 sech 2 a 2 ^x - x 2h . Specifically, c 1 = 0.3, c 2 = 0.1, x 1 = 0.4, x 2 = 0.8, a 1 = 0.5 c 1 /o ,
and a 2 = 0.5 c 2 /o . With this initial condition, the equation
simulates the collision of 2 waves of different magnitudes traveling from different locations. These 2 waves gradually lose
their magnitudes while they approach each other, and then col-

In this section, we provide visualization and discussion on how
the mixing coefficients change during the evolutionary process,
under the proposed mixture model-based adaptive transfer.
Taking the 2D projectile motion as example, the target problem is to solve the projectile motion under the effect of moon
gravity g = 1.6, with an initial launch angle a 0 = 45.
Firstly, a multi-source setup is considered, where one of the
sources is less relevant (optimized search distribution for g = 9.8,

Multi-Source

Single Source
Unrelated Prior, Initial αϕ = 0.99
Related Prior, Initial αϕ = 0.99
Related Prior, Initial αϕ = 0.1
Related Prior, Initial αϕ = 0

1

0.8

Target
Source 1
(Less Relevant)
Source 2
(More Relevant)

0.6
0.4
0.2

Mixing Coefficient

Mixing Coefficient

1

0

0.8
0.6
0.4
0.2
0

0

2k

4k
6k
8k
10k
Number of Evaluation

12k

14k

0

2k

4k
6k
8k
10k
Number of Evaluation

12k

14k

(a)
Single Source

10

102

101

101
Loss

Loss

Multi-Source
2

100

10-1

Unrelated Prior, Initial αϕ = 0.99
Related Prior, Initial αϕ = 0.99
Related Prior, Initial αϕ = 0.1
Related Prior, Initial αϕ = 0

100

10-1
0

2k

4k
6k
8k
10k
Number of Evaluation

12k

14k

0

2k

4k
6k
8k
10k
Number of Evaluation

12k

14k

(b)
FIGURE 9 (a) In the multi-source plot, the history of mixing coefficient as' from all mixture model components during a tNES search process
are shown. In the single source plot, the history of mixing coefficient of a single source is shown under different initial values of a { . Their
respective convergence trends between 0 and 15,000 evaluations are shown in (b), which highlights that effective transfer neuroevolution leads
to better convergence.

28

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2021
IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021
