IEEE Geoscience and Remote Sensing Magazine - September 2022 - 203

MOTIVATION
Recent technological advances have significantly increased
the volume and distribution rate of RS data, reaching the
level of tens of terabytes on a daily basis. For that reason,
such data have become a ubiquitous source of information
for the monitoring of Earth's physical, chemical, and biological
systems, assisting with atmospheric, geological, and
oceanic research as well as hazard assessment and resource
management applications, to name a few.
Satellite RS currently drives Earth observation (EO) research
and applications. There are many operational satellites
orbiting Earth mounted with active and passive RS
sensors, providing a continuous stream of information on
various aspects of the planet's physical processes. Satellite
imagery from these sensors is characterized by its spatial,
spectral, temporal, and radiometric resolutions [1]. The
spatial resolution (or the ground sample distance) refers to the
size of a single satellite image pixel on the ground and corresponds
to the level of spatial detail that can be acquired
with this particular sensor. Spectral resolution refers to the
range of the electromagnetic (EM) spectrum (wavebands)
that the sensor acquires observations in, while temporal resolution
(or revisit time) refers to the time interval between two
consecutive image acquisitions of the same location. Finally,
radiometric resolution refers to the numerical precision or
bit depth of a single pixel. Unfortunately, due to technical
and financial constraints, there are usually tradeoffs among
these factors, and no available sensor can capture information
at the highest possible spatial and temporal resolution
across all wavebands.
Therefore, one of the hottest topics in RS is the fusion of
multisource data with the aim to combine their strengths
and enhance the resolution along the spatial, spectral, or
temporal dimension. In this particular study, we focus on
the spatial downscaling problem, which can be greatly
aided by the integration of DL methods and makes up an
essential part of the pipeline of various RS research fields,
such as land use and land cover classification [2], [3], deforestation
monitoring [4], [5], crop yield forecasting, precipitation
forecasting [6], disaster monitoring [7], [8], stream
flow monitoring [9], and many more.
Several review articles were published recently that, to
a certain extent, address the problem of image downscaling
with deep neural networks. The present study aims to
differ and, ultimately, add a methodological framework as
well as a valuable summary of the most recent literature on
enhancing the spatial resolution of satellite imagery data,
specifically, using advanced DL architectures. These DL
models are tailored to EO data with their unique and heterogeneous
spatial, temporal, and spectral characteristics,
which differ significantly from the imagery traditionally
used by the CV community.
In fact, research on CV applications has motivated the
production of valuable review articles, mainly for (nonsatellite)
image super-resolution (SR), like [10]-[15] and [16].
Our work exclusively targets the RS field and provides a
SEPTEMBER 2022 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
broader overview of methods and applications than [17]-
[19], which focus solely on pansharpening approaches, or
[20], which only examines single-image SR (SISR) nonDL
methods. Additionally, a number of noteworthy studies
[21]-[23] provide a thorough analysis of the use of DL
techniques in RS, but they are not limited to the spatial
downscaling problem and address the entire spectrum of
applications. Other review works ([1] and [24]) focus on the
state of the art of multimodal data fusion, partially addressing
image resolution enhancement without focusing on DL
techniques. Finally, a study similar to ours [25] reviews the
literature up to mid-2019, therefore missing the most recent
state-of-the-art approaches.
Indeed, the last three years have been productive for scientific
works on image downscaling with DL. For example,
while publications on RS image SR have been steadily increasing,
the ratio of studies that use DL has blown up, from
5% in 2017 to almost 40% in 2020 (Figure 1). Similarly, in
CV, publications on DL for image SR [26] have exhibited a
steady increase.
In this review article, we present the recent advancements
(up to July 2021) of spatial downscaling on satellite
imaging through DL approaches and analyze their
strengths and shortcomings. We are only interested in the
enhancement of surface reflection products and do not address
geophysical variables, such as land surface temperature
(LST), vegetation indexes, and so on.
TERMINOLOGY
Before moving forward, we need to clarify which terminology
is used in this article as far as spatial resolution
increase/decrease is concerned. In climate and meteorological
(e.g., [27]) as well as RS [28] studies, the term downscale
refers to the transition from low to high resolution,
i.e., less to more detail representation. However, in the
CV field, it is the term upscale that refers to the increase of
(spatial) resolution, and downscale refers to the decrease of
518
334
90
5
95
2017
191
18
91
2018
36
132
2019
58
147
DL for Image SR in CV
DL For Satellite Image SR in RS
Satellite Image SR in RS
FIGURE 1. The number of published papers related to image SR
for traditional and DL-based techniques for the satellite RS and CV
fields [26].
203
58
152
2020 Sept. 2021
474
IEEE Geoscience and Remote Sensing Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2022
