IEEE Geoscience and Remote Sensing Magazine - June 2021 - 89

Driven by the impressive results obtained in ML and
computer vision, it is tempting to believe that everything
can be solved using only data and algorithms. We believe
that domain knowledge and model assumptions are of
prime importance and that models must be challenged
1) to respect the reality of the physical/biological/chemical
processes governing the system under study and 2)
to be accountable-by the transparency of their internal
reasoning-for the decisions to which they lead. This is
important, especially when models are intended to be
used for actual decision making and can affect balances
of power or society-changing decisions. Later in this
article, we present ideas that are aligned with these directions
(see Table 1) and centered on the injection of
domain knowledge, but for different purposes. First we
discuss physics-aware ML, which has the goal of using
domain knowledge to restrict the solution spaces of the
models so that the outcome is physically plausible (direction
4 in Table 1). This will ensure that the physical
consistency of the solutions is maintained while avoiding
aberrant outcomes that break physics (e.g., mass and
energy conservation).
We then discuss how best to obtain human-understandable
interpretations and explanations of the inner
functioning of the models, to understand why and how
models make decisions (direction 5 in Table 1). This has the
advantage of making the model trustable and nonfalsifiable
and of avoiding situations where the right conclusions
are reached for the wrong reasons.
Explainability also enhances the potential for testing novel
hypotheses and acquiring new scientific knowledge from
the analysis of the model's functioning. Directions 4 and 5
can be combined: they use domain knowledge in various
ways with different goals. The transparency of the models'
weights is not absolutely necessary at this stage, as interpretations
can be achieved by the analysis of inputs (e.g., Local
Interpretable Model-agnostic Explanations, i.e., LIME [23])
and physics awareness realized by modified loss functions.
Yet, as mentioned previously, science is about understanding
the world in which we live, not just approximating
it. We argue that, without learning causal relationships
from observational data and assumptions, this ambitious
goal of understanding the Earth system will not be possible
(direction 6 in Table 1). In this case, learning of cause-andeffect
relationships is a mix of the previous ingredients, as
domain knowledge is needed to design the model in such
a way that it can reveal (maybe novel) cause-and-effect
relationships that can be then explained using domain
TABLE 1. A SUMMARY OF THE SIX RESEARCH DIRECTIONS PRESENTED IN THIS ARTICLE.
DIRECTION IN A NUTSHELL
1
Go beyond recognition
toward induction, deduction,
spatial and temporal
reasoning, and structural
inference.
2
Think beyond the raster and
consider all the possible
inputs and sources of supervision,
in particular, geotagged
social media data.
3
Query the world by asking
questions about images and
create descriptions.
4
Make models learned
using deep neural networks
consistent with domainspecific
knowledge, like
equations from physics.
5
Enhancing interpretability
and explainability to
understand processes in ML
models in a better way.
6
Learn cause-and-effect
relationships-not just correlations-from
observations and
assumptions about the
underlying generating process
and system.
REFERENCES CURRENT ISSUES
[6], [7]
Missing or very limited
benchmarks and novel tasks as
well as reasoning models; the
interpretability is unsolved.
[8]-[10]
The presence of data set biases
and of label noise; a spatiotemporal
mismatch between data
sources and scalability, with an
increasing number of sources.
[11], [12]
Simplistic language model,
limited choice of thematic
interactions, and a lack of
large-scale infrastructure.
[1], [13]-[16] Networks' outputs are not
physically consistent; networks
are often used as emulators of
simulations but do not explore
beyond current simulators'
constraints: they cannot
discover new physical rules.
[17], [18]
A lack of human-understandable
interpretation, with a
tendency toward confirmation
bias (e.g., with attention maps)
[19]-[22]
Models cannot work with
unevenly sampled time series
or nonstationary/noisy
processes; they extrapolate
poorly.
10 YEARS FROM NOW
Intelligent systems linking meaningful
transformation of entities, e.g., over space
or time, and deriving knowledge as the
way people understand the visual world
and its processes.
Systems that use a wide variety of sources
to enable a fine-grained understanding of
the world, all with minimal human effort
required for data set building and system
design.
Visual search engines have an understanding
of questions about images and are able
to adapt to different types of requests and
are usable for everyone.
Systems trainable with much fewer data
because they constrain output space
via physical knowledge; systems that
learn a new hypothesis for new science
generation.
Models that are more understandable and
therefore more reliable and trustworthy;
models that can be queried (and
challenged) by humans about their inner
reasoning.
Machines that automatically blend domain
knowledge, observational data, and assumptions
to learn the causal graph and
generate causal-narrative explanations of
the problem.
JUNE 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
89
IEEE Geoscience and Remote Sensing Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2021
