IEEE Computational Intelligence Magazine - May 2021 - 27

MAY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

27

10-8

10-7

10-6

10-5

10-4

10-3

)

0

1

2

3

4

4

x

ν=0
Linearized
Burgers'

6

u (t, x

5

6

5

2

4

6
(d)

x
(a)

3

2

ν = 0.001
Linearized
Burgers'

0

SGD (ADAM)

2
0

4

0

2

0

-1

ν = 0.01
Linearized
Burgers'

x

t NES

1

ν = 0.1
Linearized
Burgers'

2
1.5

x

0

-1

1
x

0

-1

0.002

0.004

0.006

0.008

0.01

0.012

0.014

2
1.75
1.5
1.25
1
t
0.75
0.5
0.25
0

(g)

ν = 0.05
Nonlinear
Burgers'

ν = 0.1
Nonlinear
Burgers'

t NES Transfer From Other ν

2
(e)

ν = 0.01
Nonlinear
Burgers'

SGD (ADAM)
xNES

1

t NES

1 0.5 0 -0.5 -1-1.5
x
(b)

SGD (ADAM)

ν = 0.005
Nonlinear
Burgers'

0

0.5
0.25

0.002

0.75

1

1.25

1.5

1.75

2

2

1
0.8
0.6
0.4
0.2
0

0.004

0.006

0.008

0.01

0.012

0.014

5
4
3
2 t
1
0
u (t, x

1
0.8
0.6
0.4
0.2
0

2

1

x

0.5

0 1.5

ν = 0.001
KdV

(f)

SGD (ADAM)

(c)

1

0.5

0

ν = 0.0015
KdV

x

t NES

1.4 1.2 1 0.8 0.6 0.4 0.2
0

ν = 0.0005
KdV

0
1.5

0.25

0.5

0.75

1

1.25

1.5

1.75

x

0.002

0.004

0.006

0.008

0.01

0.012

0.014

2
1.75
1.5
1.25
1
t
0.75
0.5
0.25
0

FIGURE 8 (a)-(c) Sample solution to transient PDE problems generated by tNES. (d)-(f) Mean absolute residual maps (aggregated from both differential equation and initial condition) from 20
SGD and tNES runs. They are evaluated on a test set of 10k uniform grid points across respective problem domain. Results show that tNES performs better than SGD at regions near initial state at
t = 0 and also regions with steep gradients for all 3 problems. (g) The distributions of optimized loss on a test set given by tNES, xNES and SGD are compared for all transient PDE problems. For
the linearized Burgers' equation problems, the same source problem o = 0.02 is used by the tNES. For the nonlinear Burgers' equation problems o = 0.005 & 0.01, source problems came from
o = 0.006 & 0.012; for problems o = 0.05 & 0.1, source problems came from o = 0.06 & 0.12. For the KdV equation problem o = 0.0005, source problems came from o = 0.0004 & 0.0008; for
problem o = 0.0010, source problems came from o = 0.0008 & 0.0016; for problem o = 0.0015, source problems came from o = 0.0012 & 0.0016. The Friedman tests conclude a difference in
performance between three optimization algorithms at 5% significance level. In particular, the tNES outperforms xNES and SGD (ADAM) at 5% significance level (as per the Mann-Whitney rank
test) across all the problems considered in this example.

Optimized Loss

t

)
u (t, x

)

t

10
8
6
4

t



IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021

Contents
IEEE Computational Intelligence Magazine - May 2021 - Cover1
IEEE Computational Intelligence Magazine - May 2021 - Cover2
IEEE Computational Intelligence Magazine - May 2021 - Contents
IEEE Computational Intelligence Magazine - May 2021 - 2
IEEE Computational Intelligence Magazine - May 2021 - 3
IEEE Computational Intelligence Magazine - May 2021 - 4
IEEE Computational Intelligence Magazine - May 2021 - 5
IEEE Computational Intelligence Magazine - May 2021 - 6
IEEE Computational Intelligence Magazine - May 2021 - 7
IEEE Computational Intelligence Magazine - May 2021 - 8
IEEE Computational Intelligence Magazine - May 2021 - 9
IEEE Computational Intelligence Magazine - May 2021 - 10
IEEE Computational Intelligence Magazine - May 2021 - 11
IEEE Computational Intelligence Magazine - May 2021 - 12
IEEE Computational Intelligence Magazine - May 2021 - 13
IEEE Computational Intelligence Magazine - May 2021 - 14
IEEE Computational Intelligence Magazine - May 2021 - 15
IEEE Computational Intelligence Magazine - May 2021 - 16
IEEE Computational Intelligence Magazine - May 2021 - 17
IEEE Computational Intelligence Magazine - May 2021 - 18
IEEE Computational Intelligence Magazine - May 2021 - 19
IEEE Computational Intelligence Magazine - May 2021 - 20
IEEE Computational Intelligence Magazine - May 2021 - 21
IEEE Computational Intelligence Magazine - May 2021 - 22
IEEE Computational Intelligence Magazine - May 2021 - 23
IEEE Computational Intelligence Magazine - May 2021 - 24
IEEE Computational Intelligence Magazine - May 2021 - 25
IEEE Computational Intelligence Magazine - May 2021 - 26
IEEE Computational Intelligence Magazine - May 2021 - 27
IEEE Computational Intelligence Magazine - May 2021 - 28
IEEE Computational Intelligence Magazine - May 2021 - 29
IEEE Computational Intelligence Magazine - May 2021 - 30
IEEE Computational Intelligence Magazine - May 2021 - 31
IEEE Computational Intelligence Magazine - May 2021 - 32
IEEE Computational Intelligence Magazine - May 2021 - 33
IEEE Computational Intelligence Magazine - May 2021 - 34
IEEE Computational Intelligence Magazine - May 2021 - 35
IEEE Computational Intelligence Magazine - May 2021 - 36
IEEE Computational Intelligence Magazine - May 2021 - 37
IEEE Computational Intelligence Magazine - May 2021 - 38
IEEE Computational Intelligence Magazine - May 2021 - 39
IEEE Computational Intelligence Magazine - May 2021 - 40
IEEE Computational Intelligence Magazine - May 2021 - 41
IEEE Computational Intelligence Magazine - May 2021 - 42
IEEE Computational Intelligence Magazine - May 2021 - 43
IEEE Computational Intelligence Magazine - May 2021 - 44
IEEE Computational Intelligence Magazine - May 2021 - 45
IEEE Computational Intelligence Magazine - May 2021 - 46
IEEE Computational Intelligence Magazine - May 2021 - 47
IEEE Computational Intelligence Magazine - May 2021 - 48
IEEE Computational Intelligence Magazine - May 2021 - 49
IEEE Computational Intelligence Magazine - May 2021 - 50
IEEE Computational Intelligence Magazine - May 2021 - 51
IEEE Computational Intelligence Magazine - May 2021 - 52
IEEE Computational Intelligence Magazine - May 2021 - 53
IEEE Computational Intelligence Magazine - May 2021 - 54
IEEE Computational Intelligence Magazine - May 2021 - 55
IEEE Computational Intelligence Magazine - May 2021 - 56
IEEE Computational Intelligence Magazine - May 2021 - 57
IEEE Computational Intelligence Magazine - May 2021 - 58
IEEE Computational Intelligence Magazine - May 2021 - 59
IEEE Computational Intelligence Magazine - May 2021 - 60
IEEE Computational Intelligence Magazine - May 2021 - 61
IEEE Computational Intelligence Magazine - May 2021 - 62
IEEE Computational Intelligence Magazine - May 2021 - 63
IEEE Computational Intelligence Magazine - May 2021 - 64
IEEE Computational Intelligence Magazine - May 2021 - 65
IEEE Computational Intelligence Magazine - May 2021 - 66
IEEE Computational Intelligence Magazine - May 2021 - 67
IEEE Computational Intelligence Magazine - May 2021 - 68
IEEE Computational Intelligence Magazine - May 2021 - 69
IEEE Computational Intelligence Magazine - May 2021 - 70
IEEE Computational Intelligence Magazine - May 2021 - 71
IEEE Computational Intelligence Magazine - May 2021 - 72
IEEE Computational Intelligence Magazine - May 2021 - 73
IEEE Computational Intelligence Magazine - May 2021 - 74
IEEE Computational Intelligence Magazine - May 2021 - 75
IEEE Computational Intelligence Magazine - May 2021 - 76
IEEE Computational Intelligence Magazine - May 2021 - 77
IEEE Computational Intelligence Magazine - May 2021 - 78
IEEE Computational Intelligence Magazine - May 2021 - 79
IEEE Computational Intelligence Magazine - May 2021 - 80
IEEE Computational Intelligence Magazine - May 2021 - 81
IEEE Computational Intelligence Magazine - May 2021 - 82
IEEE Computational Intelligence Magazine - May 2021 - 83
IEEE Computational Intelligence Magazine - May 2021 - 84
IEEE Computational Intelligence Magazine - May 2021 - 85
IEEE Computational Intelligence Magazine - May 2021 - 86
IEEE Computational Intelligence Magazine - May 2021 - 87
IEEE Computational Intelligence Magazine - May 2021 - 88
IEEE Computational Intelligence Magazine - May 2021 - 89
IEEE Computational Intelligence Magazine - May 2021 - 90
IEEE Computational Intelligence Magazine - May 2021 - 91
IEEE Computational Intelligence Magazine - May 2021 - 92
IEEE Computational Intelligence Magazine - May 2021 - 93
IEEE Computational Intelligence Magazine - May 2021 - 94
IEEE Computational Intelligence Magazine - May 2021 - 95
IEEE Computational Intelligence Magazine - May 2021 - 96
IEEE Computational Intelligence Magazine - May 2021 - 97
IEEE Computational Intelligence Magazine - May 2021 - 98
IEEE Computational Intelligence Magazine - May 2021 - 99
IEEE Computational Intelligence Magazine - May 2021 - 100
IEEE Computational Intelligence Magazine - May 2021 - Cover3
IEEE Computational Intelligence Magazine - May 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com