IEEE Geoscience and Remote Sensing Magazine - December 2021 - 151

for semantic annotation, which have been proposed for
high-resolution optical data [93], have not been explicitly
explored for classification tasks using SAR data. Furthermore,
specific metric-learning approaches to enhance
class separability [94] can be adopted for SAR imagery to
improve overall classification accuracy. Finally, one of ML's
important fields, AutoML, which had not been extensively
exploited by the remote sensing community, has found an
application in PolSAR image classification [52].
OBJECT DETECTION
Although various characteristics distinguish SAR images
from optical red-green-blue (RGB) images, the SAR object
detection problem is still analogous to optical image
classification and segmentation in the sense that feature
extraction from raw data is always a prior and crucial step.
Hence, given the success in the optical domain, there is
no doubt that deep learning is one of the most promising
ways to develop state-of-the-art SAR object detection algorithms.
The majority of the earlier work related to SAR
object detection using deep learning consists of taking
successful deep learning methods for optical object detection
and applying them with minor tweaks to military
vehicle detection [the Moving and Stationary Target Acquisition
Recognition (MSTAR) data set] and ship detection
with custom data sets. Even small networks are easily
able to achieve more than 90% test accuracy for most of
these tasks.
The first attempt at military vehicle detection can be
found in [7], where Chen et al. used an unsupervised sparse
autoencoder to generate convolution kernels from random
patches of a given input for a single-layer CNN, which generated
features to train a Softmax classifier for categorizing
military targets in the MSTAR data set [96]. The experiments
in [7] showed great potential for applying CNNs to
SAR target recognition. With this discovery, Chen et al. [97]
proposed A-ConvNets, a simple five-layer CNN that was
able to achieve state-of-the-art accuracy of approximately
99% on MSTAR. Following this trend, more and more authors
applied CNNs to MSTAR [37], [98], [99]. Morgan [37]
successfully applied a modestly sized, three-layer CNN to
MSTAR, and, building on that work, Wilmanski et al. [100]
investigated the effects that initialization and optimizer selection
had on the final results. Ding et al. [98] investigated
the capabilities of a CNN model combined with domainspecific
data augmentation techniques (e.g., pose synthesis
and speckle adding) in SAR object detection. Furthermore,
Du et al. [99] proposed a displacement- and rotation-insensitive
CNN and claimed that data augmentation using
training samples is necessary and critical during the preprocessing
stage.
On the same data set, instead of treating a CNN as an
end-to-end model, Wagner [101] and, similarly, Gao [102]
integrated a CNN and an SVM by first using a CNN to
extract features and then feeding the features to an SVM
for final prediction. Specifically, Gao et al. [103] added a
DECEMBER 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
class of separation information to the cross-entropy cost
function as a regularization term, which they showed explicitly
facilitated intraclass compactness and separability
and improved the quality of the extracted features. More
recently, Furukawa [104] proposed VersNet, an encoder-
decoder-style segmentation network, to not only identify
but localize multiple objects in an input SAR image. Moreover,
Zhang et al. [95] proposed an approach based on multiaspect
image sequences as a preprocessing step. They accounted
for backscattering signals from different viewing
geometries, followed by feature extraction through Gabor
filters and dimensionality reduction; they eventually fed
the results to a bidirectional LSTM model for the joint recognition
of targets. This SAR awareness-trial-repeat framework
is presented in Figure 4.
Ship detection is another SAR task. Early studies of applying
deep learning models to ship detection [105]-[109]
mainly consisted of two stages: first, cropping patches from
the whole SAR image and then identifying whether cropped
patches belonged to target objects by using a CNN. Because
of fixed patch sizes, these methods were not robust enough
to accommodate variations in ship geometry, such as size and
shape. This problem was overcome by using region-based
CNNs [110], [111], with the creative use of skip connections
and feature fusion techniques in later literature. For example,
Li et al. [112] fused features of the last three convolution layers
before feeding them to a region proposal network (RPN).
Kang et al. [113] introduced a contextual region-based network
that fused features from different levels. Meanwhile, to
make the most use of features of different resolution, Jiao
et al. [114] densely connected each layer to subsequent ones
and fed features from all the layers to a separate RPN to generate
proposals; in the end, the best proposal was chosen
based on an intersection-overunion score.
In more recent works on SAR object detection, scientists
have tried to explore many other interesting ideas to
complement current efforts. Dechesne et al. [115] proposed
a multitask network that simultaneously learned to detect,
classify, and estimate the length of ships. Mullissa et al. [84]
showed that CNNs can be trained directly with complexvalued
SAR data; Kazemi et al. [117] performed object classification
using an RNN-based architecture directly on received
SAR signals instead of processed SAR images; and
Rostami et al. [118] and Huang et al. [119] explored knowledge
transfers and transfer learning from other domains to
the SAR arena for object detection.
Perhaps one of the more interesting recent works in
this application area relates to building detection, by
Shahzad et al. [120]. The authors tackle the problem of
very-high-resolution (VHR) SAR building detection using
an FCN [121] architecture for feature extraction, followed
by a conditional random fields RNN [122], which helps
give similar weights to neighboring pixels. This architecture
produced building segmentation masks with up to
93% accuracy. An example of the detected buildings can
be seen in Figure 5, where Figure 5(a) is the amplitude of
151
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
