IEEE Geoscience and Remote Sensing Magazine - December 2021 - 150

Apart from these single-image classification schemes using
CNNs, the use of SAR image time series for crop classification
has been shown in [40] and [82]. The authors of both
papers experimented with using RNN-based architectures
to exploit the temporal dependency of multitemporal SAR
images to improve classification accuracy. A unique approach
for tackling PolSAR classification was recently proposed
in [52], where, for the first time, the authors utilized
an AutoML technique to find the optimum CNN architecture
for each data set. The approach takes into account
the complex nature of PolSAR images, is cost effective, and
achieves high classification accuracy [52].
Most of the aforementioned methods rely primarily on
preprocessing and transforming raw, complex-valued data
into features in the real domain and then inputting the
data in a common CNN, which constrains the possibility of
directly learning features from raw information. To tackle
this problem, Zhang et al. [83] proposed a novel complexvalued
CNN (CV-CNN) specifically designed to process
complex values in PolSAR data, i.e., the off-diagonal elements
of a coherency or covariance matrix. The CV-CNN
not only takes complex numbers as inputs but also employs
complex weights and complex operations throughout
different layers. A complex-valued backpropagation
algorithm was also developed for CV-CNN training. Other
notable complex-valued deep learning approaches for classification
using PolSAR images can be found in [84]-[86].
Differing from the previously mentioned works, which exploit
the complex-valued nature of SAR images in PolSAR
image classification, Huang et al. [87] recently proposed
a novel deep learning framework called the Deep SAR-Net
for land use classification focusing on feature extraction
from single-polarimetric complex SAR images. The authors
perform a feature fusion based on spatial features learned
from intensity images and time-frequency features extracted
from the spectral analysis of complex SAR images. Since
the time-frequency features are highly relevant for distinguishing
different backscattering mechanisms within SAR
images, they gain accuracy in classifying man-made objects
compared to the use of typical CNNs, which focus only on
spatial information.
Although not completely related to terrain surface classification,
it is also worth mentioning that the combination
of SAR and PolSAR images with feed-forward neural networks
has been extensively used for sea ice classification.
This topic is not treated any further in this section, and the
interested reader is referred to [88]-[92] for more information.
Similar to the polarimetric signature, InSAR coherence
provides information about physical scattering properties.
In [35], interferometric volume decorrelation is used
as a feature for forest/nonforest mapping together with radar
backscatter and the incidence angle. The authors used
bistatic TerraSAR-X Add-On for Digital Elevation Measurement
data, where temporal decorrelation can be neglected. They
compared different architectures and concluded that CNNs
outperformed the random forest and that the U-Net [32]
proved best for this segmentation task.
To summarize, it is apparent that deep learning-based
SAR and PolSAR classification algorithms have advanced
considerably in the past few years. Although, at first, the
emphasis was on low-rank representation learning using
SAEs [67] and its modifications [70], later research
focused on a multitude of issues relevant to SAR imagery,
such as taking into account speckle-preserving [68], [70]
spatial structures [72] and their complex nature [83]-[85],
[87]. It can also be seen that the labeled data scarcity challenge
has driven researchers to use semisupervised learning
algorithms [86], although weakly supervised methods
PolSAR Data
Preprocessing
Dual-CNN Feature Extraction and Classification
Convolution 61,
500 at 3 × 3
Matrix T
Six-Channel
CNN
Dual-CNN
Pauli RGB
CNN
Pauli
Decomposition
Pauli RGB
Pooling
Convolution 31,
500 at 3 × 3
Pooling
Convolution 32,
100 at 3 × 3
FC3_84
FC3_200
FIGURE 3. The architecture of the dual-branch deep CNN (the Dual-CNN) for PolSAR image classification proposed in [72]. FC: fully connected;
RGB: red-green-blue.
150
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
Six Channels
A
D
B
E
C
F
Convolution 62,
100 at 3 × 3 FC6_200
Pooling
Pooling
FC6_84
FC6_14
Softmax
Class 1
Class 1
Class N

IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021