IEEE Geoscience and Remote Sensing Magazine - December 2023 - 13

DATA AUGMENTATION
Data augmentation is a simple yet widely applied approach
for increasing dataset diversity. As for the scale variation
problem in multiscale object detection, image scaling is a
straightforward and effective augmentation method. Zhao
et al. [35] fed multiscale image pyramids into multiple networks
and fused the output features from these networks to
generate multiscale feature representations. In [36], Azimi
et al. proposed a combined image cascade and feature pyramid
network (FPN) to extract object features on various
scales. Although image pyramids can effectively increase
the detection performance for multiscale objects, the inference
time and computational complexity are severely
increased. To tackle this problem, Shamsolmoali et al. [37]
designed a lightweight image pyramid module (LIPM). The
proposed LIPM receives multiple downsampling images to
generate multiscale feature maps and fuses the output multiscale
feature maps with the corresponding scale feature
maps from the backbone. Moreover, some modern data
augmentation methods (e.g., Mosaic and Stitcher [38]) also
show remarkable effectiveness in multiscale object detection,
especially for small objects [39], [40], [41].
MULTISCALE FEATURE REPRESENTATION
Early studies in RSOD usually utilized the last single feature
map of the backbone to detect objects, as illustrated
in Figure 5(a). However, such single-scale feature map prediction
limits the detector's ability to handle objects with a
wide range of scales [42], [43], [44]. Consequently, multiscale
feature representation methods have been proposed
and become an effective solution to the huge object scale
variation problem in RSOD. The current multiscale feature
representation methods are mainly divided into three
streams: multiscale feature integration, pyramidal feature
hierarchy, and FPNs.
MULTISCALE FEATURE INTEGRATION
Convolutional neural networks (CNN) usually adopt a
deep hierarchical structure, where different level features
have different characteristics. The shallow-level features
usually contain fine-grained features (e.g., points, edges,
and textures of objects) and provide detailed spatial location
information, which is more suitable for object localization.
In contrast, features from higher-level layers show
stronger semantic information and present discriminative
information for object classification. To combine the information
from different layers and generate a multiscale representation,
some researchers introduced multilayer feature
integration methods that integrate features from multiple
layers into a single feature map and perform the detection
on this rebuilt feature map [45], [46], [47], [48], [49], [50],
[51], [52]. Figure 5(b) exhibits the structure of multilayer
feature integration methods.
Zhang et al. [48] designed a hierarchical robust CNN to
extract hierarchical spatial semantic information by fusing
multiscale convolutional features from three different layers
DECEMBER 2023 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
13
Prediction
Prediction
Prediction
Prediction
Prediction
(a)
(c)
Prediction
Prediction
Prediction
Prediction
Prediction
Top-Down Pathway
(b)
(d)
(f)
FIGURE 5. Single-scale feature representation and six paradigms for multiscale feature representation. (a) Single-scale feature representation. (b) Multiscale feature integration. (c) Pyramidal feature
hierarchy. (d) FPNs. (e) Top-down and bottom-up. (f) Cross-scale feature balance.
(e)
CrossScale
Balance
Prediction
Prediction
Prediction
Prediction
Prediction
Prediction
Prediction
Prediction
Bottom-Up
Pathway
Integration
IEEE Geoscience and Remote Sensing Magazine - December 2023

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2023
