Signal Processing - July 2017 - 75

Table 9. The results of LR-DM-PMC [24] (localization example).
MSE

1.14

1.52

0.77

0.77

0.79

2.91

1.01

1.24

1.26

1.44

1.32

1.49

N

5

5

5

10

10

10

50

50

100

100

500

1,000

J

50

100

10

10

5

200

5

10

5

10

5

5

K

40

20

200

100

200

5

40

20

20

10

4

2

4

E

S = NTM = 10

Range

MMSE = 0.77 ___________ Maximum MSE = 2.91

GR-DM-PMC showing a more robust behavior with respect
to the parameter choice than LR-DM-PMC. AMIS provides
very good results, although it shows some sensitivity with
respect to the choice of the initial scale parameter, v 1. Note
that LAIS provides slightly worse results than AMIS but also
shows less sensitivity with respect to the parameter choice
and outperforms the performance of N independent parallel MH chains. Finally, Figure 6 shows the evolution of the
estimators of AMIS ^ J = 300, K = 200h and standard PMC
^ N = 1,000, J = 100 h as functions of the number of iterations,
j, in one specific run.

Learning hyperparameters for GP regression models
GPs are a modern machine-learning approach to solving
regression problems [56]. Given a covariance kernel function, learning its hyperparameters is the key to attain accurate performance. In this section, we test the different AIS
schemes for estimating the hyperparameters of a GP regression model.
Let us assume that we have a set of observed data pairs,
{y i, z i} iP= 1 with y i ! R and z i ! R L, and let us denote the
corresponding P # 1 output vector as y = [y 1, f, y P] <
and the L # P input matrix as Z = [z 1, f, z L]. We address
the problem of inferring the unknown function f that links
the variables y and z. Specifically, the assumed model is
y = f (z) + e, where e ~ N (e; 0, v 2) and f (z) is a realization
of a GP, f (z) ~ GP (n (z), l (z, r)) with z, r ! R L, n (z) = 0,
and the kernel function has the form
l (z, r) = exp e -

L

/

,=1

(z , - r,) 2
o.
2a 2

[We point out that f (·) in this section has nothing to do with
the test function used previously in the article.] Given these
assumptions, the vector f = 6 f (z 1), f, f (z P)@< is distributed
as p ^f | Z, a, lh = N ^f; 0, K h, where 0 is a P # 1 null vector,
and [K] ij: = l (z i, z j) for all i, j = 1, f, P is a P # P matrix.
Therefore, d x = 2, and the vector containing the hyperparameters of the model is x = [x 1 = a, x 2 = v] ! R 2, where a is
the hyperparameter of the kernel function in (31), and v is the
standard deviation of the observation noise. In this experiment,
we focus on the marginal posterior density of the hyperparameters [56], ru (x | y, Z, l) ? p ^y | x, Z, lh p (x), which can be
evaluated analytically, but we cannot compute integrals involving it. Considering a uniform prior p (x) over [0.01, 20] 2, and
because p ^y | x, Z, lh = N ^y; 0, K + v 2 Ih, we have
log 6r (x | y, Z, l)@
= - 1 y < (K + v 2 I) - 1 y - 1 log 6det ^K + v 2 Ih@,
2
2

(32)

Table 10. The results of AMIS [21] (localization example).

(31)

MSE (v 0 = 1)

0.80

0.72

0.75

0.76

0.88

1.29

MSE (v 0 = 2)

1.53

1.48

1.42

1.29

1.48

1.71

N

1

J

200

100

50

20

10

5

K

50

100

200

500

1,000

2,000

E

S = TM = 10 4

Range

MMSE = 0.72 ___________ Maximum MSE = 1.71

Table 11. The results of LAIS [23] (localization example).
MSE

1.91

1.52

1.14

1.11

1.10

1.06

1.29

1.25

1.26

1.30

1.41

N

1

2

5

5

10

10

100

100

100

200

103

500

250

500

250

500

10

25

50

25

5

7

3

3

1

3

1

J

5·10

K

1

3

9
3

3

S

5·10

E

S + NT = NT (M + 1) = 10 4

Range

MMSE = 1.06 ___________ Maximum MSE = 1.91

9·10

8,750

7,500

7,500

5·10

IEEE SIGNAL PROCESSING MAGAZINE

9
3

|

9·10

July 2017

3

|

7,500

5·10

1
3

5·10

1
3

5·10 3

75
Table of Contents for the Digital Edition of Signal Processing - July 2017
