IEEE Geoscience and Remote Sensing Magazine - June 2022 - 176

the whole patch), and a conditional random field was used
to smooth the results.
After these first efforts, articles started to appear applying
fully convolutional networks (FCNs) [27], providing
predictions for each pixel of the patch in one go and greatly
reducing the computational cost at the inference time
[28]-[32]. Since these pioneering works, a large number
of articles tackling semantic segmentation have been published
in the field and pushed the boundaries of the performance
on these data sets. Of notable interest are articles
that tackled issues such as the integration of prior knowledge
[33], edge information [34], and invariances [35] as
well as the explicit inclusion of spatial reasoning [36].
Other works looked at methods to fuse the multiresoluDL
tion
bands acquired by most VHR satellite sensors, such as
panchromatic (PAN) and multispectral (MS) images [37]-
[39]. Bergado et al. [37], introduced a multiresolution FCN,
called the fusion network (FuseNet), to perform an end-toend
image fusion and land cover classification (Figure 2).
This architecture, tailored to VHR satellite data characteristics,
resulted in higher performance
than methods based on pan-sharpening.
Contextual label information is included
in ReuseNet [40], a fully convolutional
recurrent network able to learn
the contextual label-to-label dependencies
that are commonly captured by
techniques based on conditional random
fields.
Going beyond the human design
of the CNN architecture, Wang et al.
[43] investigated a neural architecture
search (NAS) approach [41], [42] to
automatically design the CNN for the
classification of VHR images. Unlike
other NAS methods based on reinforcement
learning or evolutionary algorithms
over a discrete and nondifferentiable
search space, their framework
uses a gradient-based method to optimize
both architecture and model parameters
[44]. A switchable module allows
for addressing both image classification
and semantic segmentation.
Another promising research line inFIGURE
1. The framework of the study showing the considered application domains.
MS
PAN
Encoder Decoder
Linear Projection
and Concatenation
Upsampling and
Linear Projection
Pixelwise
Prediction
vestigates the extension of DL models
to the direct prediction of regularized
vector outcomes, i.e., outputs that can
be immediately ingested in geographic
information system (GIS) environments
[45]-[48] (Figure 3). These developments
are expected to have many practical uses for building outline
delineation, road network extraction, and more, in
general, for urban planning and monitoring applications in
the context of SDG 11.
4×
4×
4×
4×
FIGURE 2. The architecture of the PAN-MS fusion network
(FuseNet) for the pixelwise classification of VHR multiresolution
images [37], [40].
176
IMAGE TIME SERIES
RNNs are a powerful method for modeling sequential data,
leading to much progress, especially in language processing
[49]-[51]. Their capability of learning long-range patterns
over time makes RNNs promising tools for a variety of tasks
in remote sensing (RS), too. One important example in the
context of SDGs is food security and the assessment of
famine risk (SDG 2), which calls for large-scale mapping of
agricultural activities. RNNs allow the learning of temporal
patterns specific to different kinds of agricultural land
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE JUNE 2022
T
e
r i t
r
k
n
u
e
c
u
o
r
e
S
e
t
w
N
y
s
r e Z
n
o
a
u
H
L
n
g
n
r
e
o
e
f
L
i
y
r
d
a
l
C
n
e
s
m
i
g
C
i
t
i
e
t
a
S
e
A
h
b
l
i
t
c
o
n
a
n
o
r
n
i
t
F
s
i
e
s
u
ll
y
N
C
o
N
n
v
R
o
l
u
t
i
o
n
a
l
C
L
n
d
o
t
a
i
-
t
u
s
T
S
r
e
m
M
e
m
o
. . .
CNNs
o
m
m
u
IEEE Geoscience and Remote Sensing Magazine - June 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2022
