IEEE Geoscience and Remote Sensing Magazine - December 2021 - 26

HANDLING COMPLEX CONTEXT
Objects of interest are always embedded in a typical context
with surrounding environments and objects. An HRRS image
usually involves a broad range of space and contains
many kinds of objects that form an intricate spatial pattern.
The complex background of the objects of interest increases
the difficulty of highly accurate detection; however, many
existing works have demonstrated that the proper use of
context information can improve the performance of detectors.
Current works on the adaptation of complex backgrounds
have been divided into two categories: 1) detection
with a suppressing background and 2) detection with
related context information.
DETECTION WITH SUPPRESSING BACKGROUND
Many early works, taking advantage of the remarkable
feature-extraction ability of the CNN model, directly applied
the models to adapt to the complex, changeable background
and learn discriminative features for HRRS image
detection [126], [195]. To effectively distinguish between
the target and background information, Xiao et al. [196] designed
an encoder-decoder network to perform paired semantic
segmentation for per-pixel prediction. The top-left
and bottom-right parts of the objects of interest are then
predicted, and the rotated minimum BB is generated as the
rotated anchor. Compared to the presented methods, this
method is more robust across different data sets.
DETECTION WITH RELATED CONTEXT INFORMATION
The remote sensing community has long acknowledged that
context information benefits the improvement of object detection.
Therefore, more work has been done to explore how
to make good use of that information. Context information
can be placed into two categories: local and global context
[21]. Local context refers to visual information such as the texture,
color, and objects in the region that surrounds the targets
to be detected. In contrast, global context employs scene
semantics as the additional information for target detection.
Existing methods focus mainly on fusing local contexts
to improve detection performance. Gong et al. [197] integrated
the context RoIs' mining layer into the detector. The
layer can extract local context features by mapping context
RoIs to multilevel feature maps. Considering the limited label
information provided by objects-especially small objects-in
the feature map, Mo et al. [149] doubled the size
of the region proposal box, with the center in the predicted
box, to incorporate the local context information and thus
improve the discriminative ability of features in recognizing
the objects. Ma et al. proposed a multimodel decision
fusion network [198], based on gated recurrent units
(GRUs) [199], in which one of the subnetworks is designed
to learn the local context of objects of interest and the object-object
relationships. GRUs are used to merge all of the
features and form discriminative-feature representation.
Bell et al. [200] developed the inside-outside network
(ION) to exploit information both inside and outside the
26
RoIs; it integrates the contextual information outside the
RoIs by using spatial recurrent neural networks. Xiao et al.
[129] fused auxiliary features within and around the RoIs
to represent the complementary information of each region
proposal for airport detection, effectively alleviating detection
problems caused by the diversity of illumination intensities
in remote sensing images. To generate accurate rotation
BBs in large-scale aerial images, Feng et al. [202] proposed
a detection network that introduced a novel sequence local
context module. It can extract local context features, thus
making the rotated BB fit the ship tightly. The accurate BB
can include the discriminative parts, such as the prow, and
exclude noise information, such as the background.
Other works have promoted the global context as additional
information. Focusing on the task of vehicle detection,
Tao et al. [158] proposed a vehicle-detection method
driven by scene context. This first classifies the input image
into different scene categories (e.g., road, parking lot,
and others) and then detects vehicles in different scenes
separately from the contextual information provided by
the prior scene. Incorporating the scene before vehicle detection
can effectively confine the region where vehicles
may be present and apply a more flexible postprocessing
strategy according to different scene types. By analyzing the
relationship of objects and scenes in remote sensing images,
Chen et al. [133] found that most of the objects appeared
in their relevant scenes. The objects have a strong
correlation with the contextual information of their scene.
Chen et al. [133] proposed a scene-contextual FPN that fuses
the global scene features into region proposal features for
training the classifier.
Both global and local contextual information is valuable,
so the fusion of the two may achieve a better performance.
Relevant work has been carried out on this approach.
Zhang and Liu et al. [169] proposed a context-aware
detection network to improve the accuracy of target detection;
this can learn the correlations of the global information
(at the scene level) and the local neighboring objects
or features (at the object level). Li and Gong [156] used a
double-channel network to fuse the local and global features
to enhance the discrimination of the feature.
FUTURE RESEARCH DIRECTIONS
Despite tremendous recent progress in small, weak object
detection, the main technologies are still primitive and
cannot satisfactorily address all the difficulties and challenges.
Our analysis shows that future research may focus
on (but should not be limited to) the following areas.
DETECTION WITH MULTISOURCE DATA FUSION
Detectors for small, weak object detection may not be
stable. The fusion of multiple sources/modalities of data,
such as 3D point clouds, lidar, and Internet data, is of great
importance for improving detection accuracy. Two critical
problems should be addressed: how to encode multisource
or multimodal data into a unified input for the detectors
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
