IEEE Geoscience and Remote Sensing Magazine - December 2021 - 192

of key variables, such as precipitation, sensible heat (SH),
and evaporation. Results confirm the validity of the method,
for which we anticipate wide use and adoption. Code
and demonstrations of the implemented algorithms and IT
measures are provided.
EARTH DATA AND INFORMATION DELUGE
Understanding spatial temporal dynamics of Earth system
models and ovservation data are fundamental to monitoring
our planet and understanding climate change [1]-[4].
We now face an information deluge from remote sensing
platforms that continuously increase the spatial, temporal,
and spectral resolution of data sources. Earth system data
come in high volumes, are heterogeneous, and are riddled
with uncertainty [5], which poses important challenges
in analysis, modeling, and understanding. The statistical
analysis of remote sensing data and model simulations requires
dealing with this large amount of heterogeneous,
multivariate, and spatiotemporal material. Copious amounts
of data do not necessarily mean large quantities of information.
For example, it is now widely acknowledged that
models are often correlated and share common traits,
features, and information content. Which features are the
most appropriate and representative? How can we best
quantify their information content in meaningful units?
Essential Earth variables and data products exhibit high
levels of redundancy in space and time. So, what is the
most appropriate space, time, or spatiotemporal scale one
should look at? The same questions arise when trying to assess
and choose the most adequate observational variable
and biogeophysical parameter for Earth monitoring.
From a purely statistical standpoint, information quantification
for Earth and climate data is difficult. IT is the appropriate
framework to study information content, uncertainty,
and redundancy [6]. The estimation of entropy and
MI for discrete and continuous random variables has been
addressed through different approaches in the statistics literature
[7]-[10]. But the IT measure estimation of multivariate
data is problematic. Some methods, such as using histograms
[6], [11] and nearest neighbors [8]-[10], can be very
limiting, as they do not scale well, do not converge to the
true measure, and show a high estimation bias [12]. However,
in the remote sensing and geosciences community, there
have been many successful application-driven approaches to
overcome this challenge. Examples include studying feature
redundancy in image classifiers [13], assessing the maximum
number of parameters that can be estimated given a set
of observations [14], remote sensing feature extraction and
weighting [15], [16], data fusion [17], image registration [18]-
[20], synthetic aperture radar (SAR) data characterization
[21], [22], and quantifying uncertainty in models and observations
[23]. However, again, these methods are applicationdriven,
and none have been tested in very-high-dimensional
scenarios, which is crucial for data characterization.
All information quantification metrics require a
good multivariate density estimator. This is especially
192
problematic in Earth observation (EO) data with moderate-
to high-dimensional problems and nonlinear feature
relations. These issues affect the classic parametric density
estimators based on the exponential family of solutions
and mixture distributions as well as nonparametric
methods based on histograms, kernel density estimation
(KDE), and k-nearest neighbors (kNNs). As an alternative
to these traditional methods, there is a new class of techniques
called neural density estimators [24], which are parameterized
neural networks that approximate densities.
They use the " change-of-variables " formula to estimate
the densities of inputs and enable one to draw samples
of input data. They have promise, as they have been successfully
used in applications related to Earth system sciences,
including inverse problems [25] and density estimation
[26].
In this article, we look at a particular class of models
in the neural density estimation family. In particular, we
introduce the Gaussianization method [27] and a generalized
algorithm called rotation-based iterative Gaussianization
(RBIG) [28]. This uses a repeated sequence of simpler
feature-wise Gaussian transformations and orthogonal
rotations until convergence. In each iteration, the total
correlation and the non-Gaussianity are reduced and
converge toward zero, that is, toward full independence.
The learned transformation toward the Gaussian domain
is invertible, which enables us to easily synthesize data
by inverting samples drawn from the Gaussian domain.
The approach is also advantageous because it enables us
to estimate IT measures, such as entropy, total correlation,
non-Gaussianity, and MI in high-dimensional data. It is
fast and easy to apply and has links to deep neural networks
[28]-[30].
MULTIVARIATE GAUSSIANIZATION
PROBABILITY DENSITY FUNCTION ESTIMATION
Most problems in signal and image processing, IT, and
machine learning involve the challenging task of multidimensional
probability density function (PDF) estimation.
A PDF, or simply a density (),p $
x X!
X
that xxp() ,0
()
the PDF (),p $
and outputs a density following the properties 1)
$!6 RD
= " ,, , N
f
and 2) that it has to sum to one,
takes an input
# p d = xx 1. In practice, we usually do not have access to
but we do have a set of (multivariate) samples
12
drawn from the generating process xx xx , to
estimate the PDF from. Accurate PDF estimation is important
because it enables us to 1) calculate the probability of
any arbitrary input data point, which accounts for the relative
likelihood that the value of the random variable will
equal the sample; 2) generate samples xx()p+l
from this
distribution, thus facilitating data synthesis, background
and support estimation, and anomaly detection; and 3)
calculate expectations for functions (or transformations)
of arbitrary form x()f given x(),
ables us to, e.g., characterize a system.
p
i.e., E fx
[( x)], which enIEEE
GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
