IEEE Geoscience and Remote Sensing Magazine - December 2021 - 20

of DL-based techniques in recent years. Deep convolutional
neural networks (CNNs) can extract high-level
feature representations of an input image and improve
classification performance. Girshick et al. [70] took the
lead, applying CNNs to object detection by developing
region-based CNN (RCNN) features. Since then, many
milestones have marked the unprecedented speed of the
development of object detection. The main milestone approaches
are reviewed in the following sections; they can
be categorized into two classes according to the presence
or absence of a proposal generation stage: two- and onestage
detection frameworks. In the next sections, existing
milestones of the two categories of detection frameworks
are introduced first, and then the advances of DL-based
detectors in small, weak object detection are reviewed.
TWO-STAGE DETECTION FRAMEWORKS
As depicted in Figure 9(a), for an input image, a two-stage
detector would first examine DL features using a pretrained
CNN architecture. Then, in the region proposal step, many
regions of interest (RoIs), i.e., regions where a target may
likely exist, would be generated. Finally, a detection head
with a classifier and a regressor would simultaneously predict
the location and category of a target for each RoI. The
critical characteristic of two-stage detection frameworks is
that they contain a prepressing component for generating
object proposals. These kinds of detectors have dominated
object recognition since the creation of RCNNs [70] due
to their remarkable detection performance on benchmark
data sets.
REGIONS WITH CNN FEATURES
The main principle of RCNNs [70] is that they first extract
a set of object proposals (candidate boxes) using a selective
search. The proposals are resized to a fixed scale and fed
into a CNN model pretrained on ImageNet [12] to extract
high-level features; for example, Visual Geometry Group
[81], a residual neural network (ResNet) [13], and ResNeXt
[82]. Then, a linear SVM classifier is used to predict the presence
of an object and the object category for each proposed
region. RCNNs have achieved remarkable improvement in
natural image object detection, but they have obvious drawbacks;
for example, the selective search strategy may generate
more than 2,000 proposal candidates for one image,
RPN
For Each Pixel Position
Whether There
Is a Target
Box Location
For Each RoI
Multiclass
Classification
BB
Regressor
Input Image
Feature Extractor
Feature
Maps
Feature Maps
With Proposal
(a)
For Each Grid
Multiclass
Classification
BB
Regressor
Input Image
Feature Extractor Feature Maps Feature Grid
(b)
FIGURE 9. The main structures of mainstream frameworks. (a) An illustration of two-stage detection frameworks (using a faster RCNN as
an example). (b) An illustration of one-stage detection frameworks (using YOLO as an example). RPN: region proposal network;
RoI: region of interest.
20
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
Classification
and Regression
FC
Output Results
RoI Region
Vector
Classification
and Regression
Output Results

IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021