IEEE Geoscience and Remote Sensing Magazine - June 2016 - 60

regression, in general, the MATLAB SimpleR toolbox is
recommended, http://www.uv.es/gcamps/software.html,
which contains several regression tools that are organized
into families (i.e., tree-based, bagging and boosting, neural
nets, kernel regression methods, and several Bayesian nonparametric models such as GPs). The toolbox is intended
for practitioners with little expertise in machine learning who may want to easily assess advanced methods in
their problems.
adVanceS in gaUSSian proceSS regreSSion
This section reviews some of the recent advances in GPR
that are especially suited to remote sensing data analysis.
Also discussed are the main aspects of design covariance
functions that capture nonstationarities and multiscale
time relations, as well as GPs that can learn arbitrary transformations of the observed variable and noise models. The
multitask and multioutput problems are also discussed.
STRUCTURED, NONSTATIONARY, AND MULTISCALE
GAUSSIAN PROCESS REGRESSION
Commonly used kernel families include the SE, periodic (Per), linear (Lin), and rational quadratic (RQ)
(see Table 1). Figure 2 shows the base kernel illustrations
and drawings from the GP prior. These base kernels can
be combined by following simple operations (i.e., summation, multiplication, or convolution) so that one may
build sophisticated covariances from simpler ones. It is
important to note that the same essential property of kernel methods applies here; therefore, a valid covariance
function must be positive semidefinite. In general, the
kernel design should rely on the information available for
each estimation problem and should strive for the most
accurate solution with the fewest number of samples.
In Figure 2, all of the base kernels are one-dimensional, but kernels over multidimensional inputs can be constructed by adding and multiplying kernels over individual
dimensions. By summing kernels, the data can be modeled
as a superposition of independent functions, possibly representing different structures. For example, in multitem-

poral image analysis, one could dedicate one kernel for the
time domain (perhaps trying to capture trends and seasonal effects) and a kernel function for the spatial domain
(equivalently capturing spatial patterns and autocorrelations). In time-series models, the sums of kernels can express the superposition of different processes that are possibly operating at different scales; changes in geophysical
variables through time often occur at different temporal
resolutions (e.g., hours or days), and this can be incorporated into the prior covariance with those simple operations.
In multiple dimensions, summing kernels gives additive
structure over different dimensions, similar to generalized additive models [11]. Alternatively, multiplying kernels allows us to account for interactions between different input dimensions or different notions of similarity.
The following section will explain how to design kernels that incorporate particular time resolutions, trends,
and periodicities.
GAUSSIAN PROCESS REGRESSION
TIME-BASED COVARIANCE
As previously stated, time is an additional and important
variable to consider in many remote sensing applications.
Signals to be processed typically show particular characteristics with time-dependent cycles and trends. One could, of
course, include time, t i , as an additional feature in the input
sample definition. This stacked approach [34] essentially
relies on a covariance function k (z i, z j) , where z i = [t i, x i]< ,
which is convenient because it does not require learning
additional hyperparameters. However, the shortcoming is
that the time relationships are naively left to the nonlinear
regression algorithm, and, hence, no explicit time-structure model is assumed. To more consistently cope with
such temporal behavior of the observed signal, one can use
a linear combination (or composite) of different kernels,
i.e., one dedicated to capturing the different temporal characteristics and the other to the feature-based relationships.
A simple strategy that is quite common in statistics and
signal processing is to rely on a tensor kernel, as in
k (z i, z j) = k (x i, x j) # k (t i, t j),

Linear

RBF

Rat. Quadratic

Periodic

(a)

(b)

figUre 2. (a) The base kernels and two random draws from a
GP with each (b) respective kernel. See Table 1 for the explicit
functional form of each kernel.

but more sophisticated structures can be adopted. The issue
here is how to design kernels that are capable of dealing
with nonstationary processes.
One possible approach is to use a stationary covariance
operating on the variable of interest after being mapped with
a nonlinear function engineered to discount such undesired
variations. This approach was used in [35] to model the
spatial patterns of solar radiation with GPR. It is also possible to adopt an SE as a stationary covariance acting on the
time variable mapped to a two-dimensional periodic space,
z (t) = [cos (t), sin (t))] < , as explained in [23],
k (t i, t j) = exp d -

|| z (t i) - z (t j) || 2
n,
2v t2

ieee Geoscience and remote sensing magazine

(6)

JUNE 2016

http://www.uv.es/gcamps/software.html

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2016