IEEE Geoscience and Remote Sensing Magazine - December 2021 - 21

increasing very significantly the computation cost and slowing
the detection speed.
SPATIAL PYRAMID POOLING NETWORK
To reduce the computational costs incurred by an RCNN, He
et al. [71], [83] proposed the spatial pyramid pooling network
(SPPNet), wherein the SPP layer is the main improvement.
Instead of requiring an input image of fixed size, the SPP layer
can generate a fixed-length feature representation regardless
of the size of the input proposals. During the detection process,
the feature maps need only be computed once from the
entire image. The SPP layer can then extract the corresponding
region of the feature maps and generate a fixed-size feature
representation for each region proposal. This significantly
speeds up detection by avoiding repeated computations of
the feature maps. SPPNet achieved speeds more than 20-times
faster than those of RCNNs. However, it is not an end-to-end
framework and can fine-tune only its fully connected layers,
thus limiting the efficiency and performance of the model.
FAST RCNN AND FASTER RCNN
In 2015, Girshick et al. [72] proposed the fast RCNN detection
framework that uses a unified neural module to
localize and recognize targets. It increases detection precision
and accelerates detection speed because it can train a
classifier and a BB regressor simultaneously. Although fast
RCNN outperforms RCNNs and SPPNet, it is restricted by
the proposal-generation strategy.
The faster RCNN framework presented by Ren et al. [73]
is a fully end-to-end framework. It breaks though the speed
bottleneck of fast RCNN by introducing a region proposal
network (RPN) that enables generated object proposals using
a CNN model. It achieved a near-real-time detection
speed and state-of-the-art accuracy. From RCNNs to Faster
RCNN, the building blocks of a detector, including region
proposal generation, feature extraction, and BB regression,
have been gradually improved and unified into an effective
learning framework.
REGION-BASED FULLY CONVOLUTIONAL NETWORK
The regionwise subnetwork for localizing and recognizing
an object in faster RCNN still needs to be applied per region
proposal (several hundred proposals per image). To address
this problem in faster RCNN, Dai et al. [74] proposed the
region-based fully convolutional network (RFCN), a fully
convolutional architecture with most of the computations
shared over the entire image. Dai et al. constructed a set of
position-sensitive score maps by using a bank of specialized
convolutional layers as the FCN output and adding a
position-sensitive RoI pooling (RoIPool) layer on top. An
RFCN with ResNet101 could achieve an accuracy comparable
to faster RCNN (often with faster running times).
MASK RCNN
Mask RCNN was presented by He et al. [84], [85] to tackle
pixelwise object-instance segmentation by extending faster
DECEMBER 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
RCNN. Mask RCNN adopts the same two-stage pipeline
with an identical first stage (RPN). In the second stage, mask
RCNN adds a branch that outputs a binary mask for each
RoI in parallel with the class prediction and box offset. The
new branch is an FCN [86], [87] on top of a CNN feature
map. To avoid the misalignments caused by the original
RoIPool layer, an RoI alignment layer was proposed to preserve
the pixel-level spatial correspondence. With a backbone
network, i.e., a ResNeXt101-feature pyramid network
(FPN), mask RCNN achieved the top results for COCO objectinstance
segmentation and BB object detection [46].
FPN
The previous examples detect objects on only the top layer
of the feature-extraction network. In some cases, this is not
suitable for localizing objects, especially small ones.
Lin et al. [88] proposed an FPN whose top-down architecture
has skip connections to the remaining all-scale feature
maps. It shows great advances for detecting objects with a
wide variety of scales and aspect ratios and has been set as
a basic building block in many recent detectors.
CHAINED CASCADE NETWORK AND CASCADE RCNN
Two-stage object detection can be considered a cascade
structure; the first detector removes large amounts of background,
and the second stage classifies the remaining regions.
Recently, a series of end-to-end learning of more
than two cascaded classifiers and regressors for generic object
detection in the chained cascade network [89] was proposed,
extended in cascade RCNN [90], and later applied
to simultaneous object detection and instance segmentation
[91]. These models have a sequence of detection heads
trained with increasing IoU thresholds. The subsequent
heads with the increasing IoU thresholds would train on
more abundant positive samples to conduct accurate detection
and avoid the problem of overfitting.
ONE-STAGE DETECTION FRAMEWORKS
Although two-stage detectors perform satisfactorily, they
are computation intensive and therefore unsuitable for scenarios
with limited storage and computational capability.
Research scholars have therefore started to design one-stage
unified detection approaches to accelerate detection speed.
As displayed in Figure 9(b), a one-stage detector directly
predicts the locations of the BB and the class probabilities
in an entire image by using a single CNN. It does not involve
the steps of region proposal generation, feature resampling,
and postclassification, but it does encapsulate all
of the computations in a single network [20].
YOU ONLY LOOK ONCE
You only look once (YOLO), presented by Joseph et al. [75],
is considered the first one-stage detector in the DL era. The
model divides the entire image into many regions then predicts
the category probabilities and BB offsets for each region
simultaneously. Two improved versions, YOLO v2 and
21
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
