IEEE Geoscience and Remote Sensing Magazine - June 2021 - 45

noise, instead of using clean-noisy image pairs. The need
for multiple acquisitions of the same scene, however, represents
a significant obstacle.
This problem does not occur in single-image techniques,
which rely on careful data modeling by assuming
spatially uncorrelated noise and statistical priors about the
data distribution. Two recent works, Noise2Self [111] and
Noise2Void [112], show that, as long as the noise is spatially
uncorrelated, it is possible to train an image restoration
model using only individual noisy images. These two
methods introduced the concept of the blind-spot network,
where a pixel is excluded from its own receptive field, as
in Figure 8. This enables the network to learn to estimate
the center pixel from its receptive field, e.g., by minimizing
the 2
, distance between the noisy pixel and its corresponding
predicted value. By excluding the center pixel from the
receptive field, the network is prevented from learning the
identity function. Clearly, this works only when the noise
is spatially uncorrelated, which may be challenging for SAR
images. Alternative single-image approaches also use losses
that act as no-reference surrogates of the supervised loss
(e.g., derived from Stein's Unbiased Risk Estimate (SURE)
[113], which acts as an estimator of the MSE with respect
to an unavailable clean image). Since all these single-image
methods introduce a number of assumptions to avoid the
need for multiple images, it remains to be seen how restrictive
the suppositions are and how they could be refined in
future works.
SELF-SUPERVISED DESPECKLING METHODS
A few works tried to extend the Noise2Noise approach to
despeckling. However, applying it in the despeckling context
is not straightforward since many acquisitions of the
same scene are required. To overcome this issue, different
solutions have been proposed. In [104] and [105], pairs of
synthetically speckled images are generated. In particular,
[104] uses optical images with synthetic speckle, so it is
unclear why this method innovates with respect to supervised
training, given that Noise2Noise comes with a performance
penalty due to having the clean data only within the
limit of infinitely many noisy realizations. Instead, [105] is
trained employing only real SAR images. In [105], an adversarial
learning framework consisting of two generators and
a discriminator is employed to produce images of the same
scene with different speckle realizations. The adversarial
training forces the distribution of the generated images to
match that of the real SAR images. The main limitation of
such solutions is that the matching is necessarily imperfect,
leading to a domain gap between the generated images employed
in training and the real images used in testing.
A different solution is presented in [107], where multitemporal
data are employed to obtain multiple images of
the same scene. After preliminary training with pairs of synthetically
speckled images, the network is fine-tuned using
a temporal series of SAR images, avoiding the domain gap
problem that occurred in the previous methods. However,
JUNE 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
temporal changes might be present in the SAR time series,
and it is necessary to carefully compensate for them. The
authors' code is available online (https://github.com/
emanueledalsasso/SAR2SAR). Another work proposing a
multitemporal approach is [114]. To account for temporal
changes in the images of a time series, a similarity measure
for each input reference pixel pair is introduced. Here, too,
the authors' code is available online (https://github.com/
ahuyzx/NR-SAR-DL). A similar approach is applied in the
context of video SAR despeckling in [115], where a registration
network first compensates for the background motion
between adjacent frames and the resulting pairs of images
are then employed to train a denoising network via the
Noise2Noise strategy.
Even though the methods presented in the preceding
propose different solutions to obtain multiple observations
of the same scene, the problem still stands, and it
represents the main limitation of the Noise2Noise approach.
The works in [109] and [41] extend the single-image
blind-spot approach of [116] to propose a self-supervised
Bayesian framework for SAR despeckling, including
noise models and priors on the conditional distribution of
the blind-spot pixel, given its receptive field. The authors
employ a whitening preprocessing and a network with a
variable blind-spot size to compensate for the autocorrelation
of the speckle process. The main limitation lies in the
assumptions that are necessarily introduced to train the
model with only single noisy observations. Careful statistical
modeling of the data and better handling of spatial
correlation could unlock further improvements. The authors'
code is available (https://github.com/diegovalsesia/
speckle2void).
FIGURE 8. The receptive field of a blind-spot network. The features
associated to the pixels in green contribute to the features of the
pixel with the red border. Notice how the pixel in red does not
contribute to its own features.
45
https://www.github.com/emanueledalsasso/sar2sar https://www.github.com/emanueledalsasso/sar2sar https://www.github.com/ahuyzx/nr-sar-dl https://www.github.com/ahuyzx/nr-sar-dl https://www.github.com/diegovalsesia/speckle2void https://www.github.com/diegovalsesia/speckle2void
IEEE Geoscience and Remote Sensing Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2021
