Signal Processing - July 2017 - 77

Table 13. The results of standard PMC [19] (GP example).
12
11
10
9

MSE

0.44

0.87

1.01

0.88

0.86

0.95

1.15

100

200

500

1,000

2,000

200

100

S = NT = 10

Range

MMSE = 0.44 ___________ Maximum MSE = 1.15

8
7

Table 14. The results of LR-DM-PMC [24] (GP example).
2

(a)

MSE

0.41

0.39

0.16

0.09

0.04

0.23

0.07

0.46

1,000

100

200

100

S = NTM = 10 4
MMSE = 0.04 ___________ Maximum MSE = 0.46

Range

particle filtering, there is enough common ground between the
two methodologies to investigate the application of particle
flows to AIS. How can the underlying principles of particle
2
flows be exploited in AIS?
In recent years, stochastic optimization methods have
5
10
15
20
(b)
seen a resurgence. One reason for this is that there are many
problems that can be formulated as optimization problems, in
FIGURE 7. (a) The posterior density r (x y, Z, l) . (b) The evolution of the
which the minimized objective function is a sum of many loss
location parameters n n, t in one specific run of LAIS with N = 5 and T = 100
functions. IS is one of a number of MC sampling-based meth(jointly with the contour plot of the posterior pdf). The starting points are
ods for stochastic optimization. It can improve the convergence
shown with x marks, whereas the final locations are depicted with circles.
rate of the optimization and reduce the stochastic variance of
the result [58]. The use of AIS for optimization raises various
of the rates of convergence of AIS methods than those of
challenging questions, including convergence to optimal soluMCMC sampling. A pitfall of IS methods is the possibility
tions and optimal values.
of using proposal pdfs with thinner tails than those of the tarA specific application of stochastic optimization is in stoget distribution, which can easily ruin any estimate from the
chastic variational Bayesian methods. These methods can be
generated data and the computed weights.
applied to complex probabilistic models and large data sets
The most important open problem of AIS, as we have
with a vast range of applications in machine learning. Recentalready alluded, is the development of AIS methods that
ly, a synthesis between variational inference and MCMC
can work accurately in high-dimensional
sampling for variational approximation has
spaces. As the dimension of x increases,
been proposed [59]. It was claimed that a
In the years to come, we
the complexity of finding good proposal
fast posterior approximation through the
expect that AIS methods
pdfs explodes (curse of dimensionality).
maximization of an explicit objective was
will find increased
One approach for resolving this problem is
accomplished. Furthermore, the proposed
use within the signal
to work with compartmentalized spaces of
method offered tradeoffs between comthe unknowns and accept that we will not
putation and accuracy. Clearly, AIS is a
processing community.
have approximations of the full joint postenatural candidate to be applied in the same
rior but instead a number of marginalized posteriors.
setting with the possibility of performing even better than
Another way of addressing high dimensionality is by parMCMC sampling.
ticle flows. This approach has been of interest in particle filFinally, in the years to come, we expect that AIS methods
tering, where samples drawn from the prior distribution are
will find increased use within the signal processing commigrated to the posterior distribution of the unknowns by solvmunity. Much of the research in this area will be driven by
ing partial differential equations [57]. Even though the probnovel applications and by models with expanded complexity.
lems we solve with AIS are different from those addressed by
There will be new applications that may even include use of
4

IEEE SIGNAL PROCESSING MAGAZINE

July 2017

Table of Contents for the Digital Edition of Signal Processing - July 2017