IEEE Geoscience and Remote Sensing Magazine - June 2022 - 216

Although vector/matrix-based methods have achieved
excellent destriping results, they may cause spectral and spatial
distortion. To alleviate this, low-rank tensor recovery approaches
have emerged in recent years [122]-[126]. For example,
Fan et al. [122] proposed a low-rank tensor recovery
model for mixed Gaussian and sparse stripe noise removal.
Chen et al. [123] presented a tensor image decomposition
framework where the
, norm is used to accommodate
21 ,
the column-wise group sparsity of a stripe. Overall, these
sparsity optimization-based techniques are explainable and
effective for all kinds of stripes. However, compared to supervised
DL-based algorithms, such as CNNs and filteringbased
methods, they are computationally expensive and
therefore not suitable for real-time applications.
DEEP LEARNING-BASED TECHNIQUES
DL-based methods, such as CNNs, have been extensively used
for various computer vision tasks. The advantage of CNNs over
conventional methods for destriping is twofold. First, thanks
to the universal approximation theory [133], which theoretically
proves that CNNs could implicitly approximate any complicated
distribution of arbitrary mixed noise, a supervised
CNN could handle arbitrary stripe noise well. Second, due to
the simple operation of the network, its test phase is extremely
fast, which makes it suitable for real-time use.
In 2017, a three-layer CNN, nonuniform correction, was
proposed by Kuang et al. [134] to remove stripe noise from
infrared images. Xiao et al. [135] increased the depth of a
network to improve its representation ability. Further, residual
learning was employed by Xiao et al. [135] for better
destriping. Moreover, Chang et al. [100] introduced a multiscale
strategy into a network to enhance representation.
Recently, Chang et al. [100] embedded a wavelet into a twostream
CNN framework to learn the internal directional
property of a stripe. A unique CNN architecture by Zhong
et al. [136] was presented; it integrates 2D and 3D convolutions,
residual learning, and supplementary gradient channels
to capture intrinsic spectral-spatial features in HSIs
and the unidirectional property of stripes. Most recently, to
overcome the lack of paired training data for DL, Song et al.
[137] proposed a novel unsupervised HSI destriping method
with a subband cycle-consistent adversarial network.
RESULTS AND COMPARISONS
We evaluate destriping methods in the following categories:
◗ Filtering-based methods: the wavelet Fourier adaptive filter
(WFAF) [105]
◗ Optimization-based methods: TV [63], unidirectional TV
(UTV) [128], statistical linear destriping (SLD) [129],
and low-rank single-image decomposition (LRSID) [119]
◗ DL-based methods: the DL stripe (DLS) [132] and twostream
wavelet enhanced U-net (TSWEU) [100].
For a data set, we use the 15 remote sensing images collected
in [100] for testing. Full-reference indices-the PSNR
and structural similarity index measure (SSIM)-and noreference
assessments-the inverse coefficient of variation
216
[128] and mean relative deviation [110]-are employed for
quantitative assessment.
Simulating different stripes involves various procedures.
We take the classical additive stripe as an example.
We generate striped image H by adding stripes S
to clean image X. The matrix coefficient is generated by
((
repmat stripe -+)
size (, ), );H 11 stripemax and stripemin are the predefined
maxmin). 12rand(, (, ))
stripe
size Hstripemin
,
hyperparameters that determine the range of the additive
stripe, and repmat and size are the MATLAB functions. Interested
readers can refer to the simulation codes for the other
stripe categories.
QUANTITATIVE EVALUATION
We show quantitative results for the methods in different
stripe categories and levels. We choose the most representative
stripes: multiplicative, periodical, nonperiodical, broken,
and mixed. It is worth noting that additive, covered,
and full types are included in the five cases. Dead lines and
oblique stripes are the only two kinds of stripes that are not
tested here since very few methods can handle such degradation.
From Table 4, we have the following observations.
First, DL-based methods consistently achieve the best performance
among all the approaches. Second, SLD handles
regular stripes well since its rank 1 constraint provides a satisfactory
fit for them. However, for broken and mixed noise,
where the assumption does not hold, SLD performance
drops rapidly. Third, the filtering-based WFAF achieves impressive
performance for periodic stripes, as they have regular
high-frequency components in the frequency domain.
Moreover, in Table 5, we report the running times
for different image sizes. We perform the experiments
in MATLAB 2017a, using a computer with an Intel i7
CPU at 3.6 GHz, a Nvidia GEFORCE RTX 1080Ti GPU,
and 32 GB of memory. The training and test times are
considerably different. We observe that the running
time of learning-based methods, including the DLS and
TSWEU, are significantly faster than that of the other
methods. Note that learning-based methods need to be
trained on large-scale data sets, which means they need
hours to days for training, while the inference/testing
time is usually very short (within 1 s). On the contrary,
handcrafted methods do not require training, while
their testing time is relatively slow (typically one or two
orders of magnitude slower than it is for deep NNs).
QUALITATIVE EVALUATION
We show several visual destriping results for the methods
in Figure 10. The main observations are: 1) most of the approaches
work well for typical stripes, such as periodical/
nonperiodical and multiplicative/additive ones. However,
for broken stripes, where the low-rank property no longer
holds for a majority of them, most of the methods fail to
satisfactorily restore the image. 2) Most of the single-imagebased
destriping techniques cannot handle random noise
well. Additionally, random noise may even have a negative
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE JUNE 2022
IEEE Geoscience and Remote Sensing Magazine - June 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2022
