IEEE Computational Intelligence Magazine - May 2021 - 20

w k " wu k = n + d k # min c 1,

	

r
m .(10)
A -1 d k

Note that the denominator A -1d k is the Mahalanobis distance
between w k and the target distribution center. As demonstrated
in Figure 4, these projected pseudo-offspring are now closer to
the target distribution, while preserving directional and fitness
information for the update. As a result, they induce a d- efinite
directional bias on the evolution of the target search distribution
without absurdly changing it in a single update.
Our formulation of transfer neuroevolution with mixture
modelling and pseudo-offspring projection is generic for probabilistic model-based search, and does not depend on a specific
algorithm. The next section presents a particular instantiation of
this method by implementing it on a variant of NES. This
transfer NES algorithm will be subsequently applied to solve
differential equations problems.

not activated, m pseudo-offspring are sampled from the target
distribution and follow the original xNES procedure. Otherwise, pseudo-offspring are sampled from the mixture model (6)
and their fitness are evaluated. Additionally, projection (10) is
applied to all pseudo-offspring from the source distribution,
after which a coordinate transformation z k = A -1^wu k - n h
maps the projected pseudo-offspring to the natural coordinates
of the target distribution (this step is required for xNES
updates). Then, fitness shaping is applied. All pseudo-offspring
(now z k) are sorted by their fitness, i.e., f ^w 1h $ g $ f ^w mh,
and the rank-based utility value:
	

uk =

max ` 0, log ` m + 1 j - log k j
2
, (11)
m
/ j = 1 max `0, log ` 2m + 1 j - log j j

is computed to replace the k-th best fitness f ^w kh. Subsequently, the algorithm follows the xNES procedure for the

C. An Instantiation: Transfer NES (tNES) Algorithm

In this section, transfer neuroevolution with a variant of NES,
namely the exponential NES (xNES) algorithm [50], is presented.
The xNES efficiently updates the mean n and the full covariance matrix R = A A T of a search distribution via a coordinate
transformation trick, to avoid trivial operations on computing
and inverting the Fisher information matrix in (5). The details of
the xNES algorithm can be found in [50]. Algorithm 1 presents
the pseudocode of our transfer implementation on xNES, which
is hereafter referred to as tNES.
The tNES algorithm requires the following inputs: fitness
function, initial search distribution, a source distribution, and
initial mixing coefficients. In the iteration loop, the tNES algorithm firstly checks for the transfer flag. If the transfer mode is

Algorithm 1 Pseudocode of tNES.

Input: optimization problem f ! R d " R, initial search
-distribution n ! R d, A = vI ! R d # d, source distribution
n { ! R d, A { ! R d # d, initial mixture coefficient a { ! 60, 1@
a r ! 1- a {
while stopping criteria not met do
  p ! a { if transfer activated else p ! 0
  for k ! " 1, 2, f, m , do

  sample z k + N ^0, I h , s k + bernoulli ^ p h
  if s k = 1 then
   w k ! A { z k + n {
    d k ! w k - n
u k ! n + d k # min c 1,
    w

High
1
µ
Directional Bias

0.5
0

Projection of PseudoOffspring From Source
Distribution

-0.5
-1

Source

Target
-1

-0.5

0

0.5

1

Low

FIGURE 4 An illustration of projecting the pseudo-offspring (marker:
cross) produced by the source distribution towards the target search
distribution. The projected pseudo-offspring are within r Mahalanobis distance from the search distribution center, while still preserving
the same directional and fitness information (indicated by color
scale). By combining them with the pseudo-offspring originating from
the target distribution (marker: round), a definite directional bias is
induced on the evolution of the target search without absurdly
changing it in a single update, thus promoting numerical stability.

20

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2021

u k - nh
    z k ! A -1 ^w
  else
   w k ! Az k + n
  end
  end
 sort " ^ z k, w k h, w.r.t. f ^w k h
 compute " u k ,

r
m
A -1 d k

  d n J ! R mk = 1 u k z k
  d A J ! R mk = 1 u k^ z k z Tk - I h
  n ! n + hn $ dn J
  A ! A $ exp ^1/ 2 $ h A $ d A J h
 if transfer activated then
   d ar J m = R mk =1 u k
   d a{ J m = R mk =1 u k

r ^w k h
a r r ^w kh + a { { ^w k h

{ ^w kh
a r r ^w kh + a { { ^w k h

   a r ! a r + h a $ d ar J m
   a { ! a { + h a $ d a{ J m
  normalize ^a r, a {h s.t. a r + a { = 1
 end
end
IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021
