IEEE Geoscience and Remote Sensing Magazine - September 2022 - 216

spectra at 30 m and a PAN band at 15-m spatial resolution
every 16 days [103], so the goal of this study is to produce
the equivalent Landsat images at 10-m spatial resolution.
The whole process can be broken down into two separate
steps. First is the self-adaptive fusion of Sentinel-2,
where the 20-m Sentinel-2 bands (11 and 12) are resampled
to 10 m using k-nearest neighbors (k-NN) interpolation
and are then concatenated with the native 10-m bands as
the input to the proposed ESRCNN model. The output is
bands 11 and 12 downscaled to 10-m resolution. Following
this is the multitemporal fusion of Landsat-8 and Sentinel-2,
where the 30-m Landsat bands (1-7) and the PAN band are
TABLE 3. SUMMARY OF THE STATE-OF-THE-ART DL MODELS FOR SSF FOR IMAGE DOWNSCALING IN RS. (Continued)
MODEL
VO+Net [125]
SC-PNN [126]
FUSION
TYPE
FUSION DATA
PAN + MS WorldView-3,
WorldView-2, and
QuickBird
PAN + MS WorldView-3, GeoEye-1,
and SPOT5
NLRNet [90]
LPPNet [118]
Scarpa et al.
[110]
Ciotola et al.
[130]
PSGAN [131]
Pan-GAN [132]
MDSSC-GAN
SAM [133]
PanColorGAN
[134]
Palsson et al.
[135]
DHSIS [136]
PFCN [137]
PAN + MS WorldView-3 and
QuickBird
PAN + MS Pavia Center, Houston,
and Los Angeles
PAN + MS GeoEye-1 and
WorldView-2
PAN + MS GeoEye-1, WorldView-2,
and WorldView-3
PAN + MS QuickBird, GaoFen-2,
and WorldView-2
PAN + MS GaoFen-2 and
WorldView-2
PAN + MS Pléiades and
WorldView-3
PAN + MS Pléiades, WorldView-2,
and WorldView-3
MS + HS Pavia Center and
IKONOS
MS + HS CAVE and Harvard
MS + HS Botswana; Washington,
D.C.; and Pavia Center
CF-BPNN [138] MS + HS AVIRIS and Pavia
Center
HyperPNN [139] MS + HS Washington, D.C.
National Mall; Moffett
Field; and Salinas
Scene
DDLPS [140]
MS + HS Moffett Field, Chikusei,
and Salinas Scene
TONWMD [141] MS + HS CAVE, Harvard, and
Pavia Center
MHF-Net [142] MS + HS CAVE, Chikusei,
Houston, and
Pavia Center
UMAG-Net [143] MS + HS CAVE and Harvard
SSR-Net [144] MS + HS Pavia Center; Botswana;
and Washington, D.C.
National Mall
CV MODEL BUILDING BLOCKS
-
Variational
optimization
-
Saliency analysis
and hybrid and
deformable
convolution
-
-
-
-
-
-
-
Residual learning and
attention mechanism
Laplacian pyramid
decomposition
Residual learning
-
-
Two discriminators:
spatial and spectral
Two discriminators:
spatial and spectral;
residual learning; and
attention mechanism
-
-
-
-
-
-
Self-supervised and
noise/color injection
-
Self-supervised and
noise injection
Residual learning
k-Means clustering
-
UPSAMPLING
FRAMEWORK ARCHITECTURE
Preupsampling CNN
Preupsampling CNN + fully
convolutional
network
Preupsampling CNN
Preupsampling CNN
Preupsampling CNN
-
CNN
Preupsampling GAN
Preupsampling GAN
Preupsampling GAN
CODE AVAILABLE/
NUMBER OF
PARAMETERS
No/-
No/-
No/-
No/-
No/-
No/-
Yes/1.88 m
No/-
Yes/-
Preupsampling GAN
No/-
Preupsampling 3D CNN
Preupsampling GAN
Preupsampling CNN
Preupsampling NN
Preupsampling CNN
No/-
Yes/-
No/-
No/-
No/-
LapSRN
-
-
Preupsampling CNN
-
Residual learning
and matrix
decomposition
-
Preupsampling CNN
Preupsampling CNN
-
-
Attention mechanism Preupsampling CNN and AE
Preupsampling CNN
-
No/-
No/-
Yes/-
No/-
Yes/-
CV Model refers to the models presented in Table 2. AE: autoencoder; AVIRIS: airborne visible/infrared imaging spectrometer; CAVE: Columbia computer vision laboratory;
NN: neural network; NLRNet: nonlocal attention residual network.
216
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE SEPTEMBER 2022
IEEE Geoscience and Remote Sensing Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2022
