IEEE - Aerospace and Electronic Systems - March 2021 - 65

Kechagias-Stamatis and Aouf
information. An overview of the LMF methods presented
is shown in Table 8.

DEEP LEARNING
During the last years, deep learning algorithms have been
gaining large attention due to their high classification
rates. An additional advantage is their end-to-end nature
that includes all three major classification stages, i.e., feature detection, description, and matching. As described in
the following subsections, in the context of SAR ATR
deep learning includes convolutional neural networks,
restricted Boltzmann machines, recurrent neural networks,
stacked autoencoders, and hybrid strategies combining
deep learning methods. An overview of the methods presented is shown in Table 9.

Convolutional Neural Networks (CNN). Gao et al. [90]
enhanced the effectiveness of their CNN by employing a
cost function during training that combines cross-entropy
with the class separability information. The latter comprises of intraclass compactness and interclass separability. Yue et al. [91] proposed a CNN algorithm that
combines both supervised and unsupervised training. Specifically, the unsupervised pipeline utilizes CNN to obtain
the class probabilities of the unlabeled samples. Then, the
class probabilities are optimized via a hard thresholding
process, which are then used to calculate the scatter matrices of the linear discriminant analysis (LDA) method.
Finally, the loss function of the supervised pipeline of the
CNN is adapted by the scatter matrices created from the
unsupervised pipeline. Chen et al. [6] and Wang et al.
[92] proposed a CNN that comprises only of sparsely connected layers (convolutional and pooling layers) neglecting the typical fully connected layers. This strategy
reduces the algorithm's free parameters and thus overfitting due to limited training images. Their all-convolutional networks (A-ConvNets) are unarguably one of the
top-performing SAR ATR techniques.
Wang et al. [93] realized the influence of specklenoise to SAR ATR and reduce its impact by introducing a
despeckling CNN as a preprocessing step applied on the
entire SAR dataset. The latter CNN comprises five convolutional layers followed by a ReLU activation function
while pooling layers are not used to preserve the feature
map size through the entire network. For completeness, a
ReLU activation function embedded within a ReLU layer
aims at increasing the nonlinearity of a CNN by applying
an individual truncation process on every input to that
layer. The advantages of ReLU against the classic tanh
activation function are the reduction in training time [94]
and incorporating a purely supervised training scheme
avoiding the need for unsupervised pretraining [95].
Kwak et al. [96] also considered the influence of specklenoise and proposed a speckle noise-invariant CNN that
MARCH 2021

employs a regularization term to minimize the feature variations caused by this type of noise. The regularization
term minimizes the CNN features between the raw and
the despeckled image variant and contributes to the final
ATR classification score in a weighted manner along with
the raw image classification output.
One subclass of CNN-based approaches involves
CNN distillation to ultimately create a compact and shallow CNN that has the classification power of deeper
CNNs. For example, Min et al. [97] extended the typical
distillation scheme into the SAR ATR domain by adopting
a student-teacher paradigm, where the teacher is a deep
18-layered CNN and the student a shallow two-layered
ternary network (weights are -1, 0, 1). During training, the
teacher's output vector after the SoftMax processing is
used to train the student's CNN. The student's loss function is the sum of the cross-entropy between the student
network output and the ground truth target label, and the
cross-entropy between the teacher network output and the
ground truth target label. The student CNN is fully trained
when its loss function minimizes. Another approach
toward lightweight designs is proposed by Zhang et al.
[98], which adapts the A-ConvNets [6], [92] to exploit a
pruning and a knowledge distillation scheme. Their
method is a three-stage process where initially the A-ConvNet is forced to a lossy compression by pruning the feature maps. Then, the CNN's accuracy is recovered via a
student-teacher type knowledge distillation concept,
where the teacher is the original un-pruned A-ConvNet
and the student its pruned variant. Trials demonstrated
that the pruned A-ConvNets completely preserves its SAR
ATR capability compared to the original A-ConvNets,
despite being 68.7 times compressed and thus 2.5 faster to
execute.
Another subclass of CNN-based approaches exploits
data augmentation to enhance CNN's classification performance by training it on more templates of various nuisances. Typical approaches create the additional templates
manually, while more recent approaches rely on a Generative Adversarial Network (GAN) [99]-[101]. For example, Wagner [102] manually generated artificial training
data that presents elastic distortion, rotation, and affine
transformations simulating a changing depression angle or
an incorrectly estimated aspect angle, aiming at increasing
the robustness against these nuisances. Similarly, Ding
et al. [103] suggested augmenting the training dataset by
generating images that include speckle noise, several
translations, and pose variations. Experimental results
demonstrate that indeed, data augmentation enhances the
ATR classification rates. Yan [104] extended the training
images by manually generating speckle-noisy samples at
different signal-to-noise ratios (SNRs), multiresolution
representations, and partially occluded images. Furthermore, the training samples are preprocessed to reduce the
interferences between the original and the augmented

IEEE A&E SYSTEMS MAGAZINE

65
IEEE - Aerospace and Electronic Systems - March 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - March 2021
