IEEE Geoscience and Remote Sensing Magazine - September 2022 - 239

Furthermore, a number of novel techniques seem to
leverage the efficiency of the underlying model, e.g., the
diagonal connectivity scheme proposed in udGAN or
the clipping-and-merging postprocessing technique and
the autoencoder loss proposed in Enlighten-GAN. Finally,
certain methods (EUSR, DWTSR, DRSEN, DSSR,
DGANet-ISE, NLASR, Shin et al. [205], and SG-FBGAN)
manage to perform better at larger scaling factors, whereas
Zhang et al. [183] provide an interesting candidate when
different distortions have taken place during the LR image
acquisition. Unfortunately, up to this point in time,
only a handful of RefSR methods have been developed,
and none seems to match the efficiency and robustness of
the SISR domain.
On the other hand, when multiple LR images can be obtained
for each training/testing sample, then MISR models
can be employed. In this family of methods, MISR-GRU
and RAMS, in particular, seem to prevail in terms of both
the resulting image quality and the number of trainable parameters.
It is worth noting that a common challenge faced
by all MISR approaches is the coregistration of the input
LR images, which is handled differently by each proposed
model, either inside the network or as a separate preprocessing
step in the pipeline. In addition, this coregistration
may incur minor shifts in the output, which, in turn, can
potentially affect the computation of the loss function during
training and encourage a blurry result. This phenomenon
has been successfully handled through the ShiftNet
module, which was proposed in HighRes-Net and, subsequently,
used in other studies. Finally, it is again proven
that attention mechanisms enhance the downscaled output
and, also, that the number and clarity of the input LR images
can greatly affect the final result.
SUPER-RESOLUTION FOR SYNTHETIC APERTURE
RADAR AND AERIAL IMAGERY
SYNTHETIC APERTURE RADAR
Most of the SAR spatial resolution enhancement techniques
related to deep neural networks use the SISR approach,
which makes the data collection, processing, and
experimentation fairly straightforward and easier compared
to optical data. However, SAR data inherently introduce
speckle noise, which few authors explicitly consider
when building SR pipelines.
Wang et al. [223] used an SISR approach by applying an
SRGAN on TerraSAR-X images after having been despeckled
using a CNN, as described in [230]. The HR image is
downsampled by a factor of four using a Gaussian kernel,
while both the generator and discriminator elements are
CNN based. The generator element produces the SR image
using the LR image, while the discriminator compares the
SR image with the HR image. The loss function comprises
a perceptual loss with a content (pixelwise MSE) and a
weighted adversarial (probability-based) component of
the discriminator.
SEPTEMBER 2022 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
Gu et al. [224] propose a transfer learning GAN-based
paradigm in dealing with speckle noise using a so-called
noise-free GAN (NF-GAN) to preserve the high-frequency
image details as much as possible. They experiment with
the horizontal-horizontal (HH) polarization channel of
Airborne SAR data. The generator element consists of a
despeckling network and the reconstruction network, while
the discriminator element is ResNet based. The despeckling
network is pretrained using optical images with speckle
noise added on them, and it uses an MSE loss. Its input is an
LR (downsampled HR version by a factor of two) noise-full
image. As with the previous case, the NF-GAN objective
function is defined by an adversarial and a pixelwise (MSE)
component. The authors train their network pipeline with
and without the despeckling component and show that the
former, indeed, works better.
Li et al. [225] tried to solve the problem of increased system
integration time and low azimuth resolution of geosynchronous
SAR (GEO SAR) using a CNN-based GAN approach.
GEO SAR is an active area of research in developing
a SAR satellite system in geosynchronous orbit, which will
significantly assist in operational disaster monitoring by
increasing the temporal resolution compared to low-Earth
orbit satellite systems. In particular, the authors generated
synthetic GEO SAR data based on advanced land observing
satellite phased array type L-band SAR (ALOS PALSAR)
characteristics. They use a dialectical GAN (Di-GAN) [231]
with the generator element comprising a U-net and the discriminator
a PatchGAN-like network. The generator takes
the LR simulated GEO SAR image as the input, whose SRproduced
image is compared with the ALOS PALSAR HR in
the discriminator. The authors claim a noticeable improvement
of the resolution, which is mostly based on a qualitative
comparison.
Cen et al. [226] propose a three-module CNN-based
network named FSRCNN for downscaling bistatic SAR
images. The first module is used for feature extraction in
various scales of the LR images. The second module adds
together the resulting feature maps that were learned from
the first module. The third module consists of a reconstruction
CNN that computes the final SR image. The authors
compare their results with bilinear, bicubic, and SRCNN
approaches using PSNR and SSIM and show an overall best
performance of the proposed FSRCNN.
Helal-Kelany et al. [232] aimed to enhance the coregistration
accuracy between two single-look complex images
of European Remote Sensing-1/2 (ERS-1/2) data. They train a
scale-invariant SR CNN (SINV CNN) model using both the
amplitude and phase, which mainly takes advantage of the
feature extraction and residual block components. Their result
is evaluated based on descriptive statistics of the coherence
between SINV CNN and sinc interpolation instead of
commonly used metrics used in CV, which may make their
output difficult to compare with other approaches.
Shen et al. [227] present a rather complete work where
they apply their technique (PSSR) to full polarimetric SAR
239
IEEE Geoscience and Remote Sensing Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2022
