IEEE Geoscience and Remote Sensing Magazine - December 2021 - 24

multiscale sliding windows with different step sizes to conduct
training with images for generating potential candidate
boxes. This method, however, is too time- and computation
consuming to meet the requirements of practical applications.
The second category is based mainly on various multiscale
features of a manual design, such as a scale-invariant
feature transform (SIFT) [130], an HoG [10], and a BoW [60].
Beril et al. [131] utilized the SIFT feature and graph theory
to detect buildings and urban areas. Shi et al. [40], [132]
combined both circle-frequency and HoG features to learn
the appearances and shapes of objects. Sun et al. [134] developed
a spatial sparse-coding BoW model to build the visual
vocabulary by clustering local features; it can effectively
fuse local and global features. However, the two categories
of methods pose difficulties when it comes to achieving satisfactory
performances for remote sensing target detection
because they all depend on handcrafted features-extracted
according to expert experience-and are not robust enough
to process complex remote sensing images.
Since 2014, many learning-based detectors that incorporate
the object proposal strategy, coupled with the remarkable
performance of DL-based features [13], [14], [81],
[135], have enabled significant improvements in the performance
of object localization and recognition [136]-[138].
Multireference and multiresolution detection, developed on
this basis, have become the two most widely used fundamental
blocks in the task of object detection [21]. The main
idea of multireference detection is to predefine a set of reference
boxes (anchor boxes) with different sizes and aspect
ratios and then to predict the detection box based on those
references. The milestone models are faster RCNN [73], RetinaNet
[77], and mask RCNN [84], [85].
Multiresolution detection detects objects with different
scales by constructing a feature pyramid at different layers
of the network. The shallow layers hold information about
small objects, while the deep layers contain information
about large objects. The main improvements are in the FPN
[88]. To detect multiscale objects, especially small ones, in
HRRS images, Guo et al. [139] and Zhang et al. [140] designed
unified multiscale detection frameworks; they used
a modified FPN as well as anchors with different scales and
aspect ratios.
Qiu et al. [141] developed an adaptive aspect ratio multiscale
network, which utilizes a multiscale feature gatefusion
subnetwork and an aspect ratio attention network
to learn the weights of different feature maps and automatically
select the appropriate aspect ratios in accordance
with the aspect ratios of objects. Wu et al. [142] introduced
multiscale and rotation-insensitive convolutional channel
features by involving two modules, the rotation-insensitive
descriptor and the multiscale aggregated descriptor. AlAlimi
et al. [143] designed a unique shallow-deep feature
extraction that employs a squeeze and excitation network
and ResNet to obtain feature maps. Deng et al. [144] addressed
the problems of scale variants by applying different
filters to several intermediate layers. Li et al. [145] proposed
24
multiscale convolutional feature fusion to detect multisensor
HRRS images using a symmetric encoder-decoder
module to extract and fuse multiscale and high-level spatial
features.
Some scholars have focused their research work on segmentation
methods. Dong and You [146], [147] utilized a
graph-segmentation algorithm. Based on multiscale saliency
maps, it is constructed to overcome the problem
of ship scale change to accurately locate candidate regions.
Kang et al. [148] designed an FCN with dense SPP for building
detection that can extract dense and multiscale features
simultaneously. Mo et al. [149] focused on generating an
anchor of the most suitable scale for each category and developed
a class-specific anchor block, which provides better
initial values for an RPN. Xie et al. [150] used multidetectors
with different sensitivities and accessed the fused
features to finish the task of target detection. Superresolution
[102] and GANs [103] have also been used to restore or
enhance the features response of small targets during the
detection process.
HANDLING INTRACLASS VARIATIONS
Objects in HRRS images vary in color, texture, and shape
feature because of the vast number of object instances and
categories as well as the influences of weather, illumination,
imaging condition, and occlusion. For real-scenario HRRS
image object detection, powerful object representations
should be extracted with robustness and discrimination.
Many recent works have been devoted to handling changes
in object variations by applying DL models to remote sensing
object detection. However, CNN models lack the ability
to be spatially invariant for generating transformations of
input data. In processing HRRS images, the performance
of these models is limited due to the intraclass variations
of objects.
Data augmentation is the most straightforward method
used to address intraclass variations, including rotation and
resizing. To some extent, these operations can make detectors
learn robustness with regard to rotation and scale, although
these methods can involve expensive training and
a massive number of model parameters. Therefore, many
attempts have been made to learn invariant CNN representations
with respect to different transformations, including
scale [151]-[153], rotation [151], [154]-[156], or both [157].
Early deformable part-based models (DPMs) [157], which
represent objects by components arranged in a deformable
configuration, were successful for generic object detection,
but these models are less sensitive to object variations in
both pose and viewpoint. Many scholars have attempted
to combine DPMs with CNNs, aiming to realize the advantages
of both [159]-[161]. To address the problem of occlusions,
deformable RoIPool [161]-[163] and deformable
convolution have been proposed to achieve more flexibility
in fixed geometric structures [27]. Another method, the application
of GANs [164], [165] to generate missing parts of
objects and context, is promising.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
