IEEE Geoscience and Remote Sensing Magazine - December 2021 - 25

HANDING THE IMBALANCE OF POSITIVE
AND NEGATIVE EXAMPLES
In essence, training a detector is a problem in imbalanced
data learning. For detectors based on a sliding
window, the imbalance between objects of interest and
backgrounds may be as extreme as 104-105 background
windows for each object [21]. For a modern detection task
with a prediction of the object aspect ratio, the imbalanced
ratios increase to greater than 106. In this case, a
vast number of negative and easy samples would guide the
training process, and the detector would achieve poor performances
for hard-to-recognize objects, especially small,
weak objects. Hard negative mining focuses on solving
the problem of imbalanced data during the training process.
Bootstrapping was a milestone technique used for
addressing the problem of a training data imbalance in
object detection, in which the training starts with a small
number of background samples to which new misclassified
backgrounds are added iteratively during the training
process [166].
Later in the DL era, detectors such as faster RCNN [73]
and YOLO [75] developed a weighted balancing method
for positive and negative samples. However, that method
cannot completely address an imbalanced data problem.
Bootstrapping was reused in DL-based detectors [76], [167].
In RefineDet [168], an anchor-refinement module is designed
to filter easy negatives. An alternative improvement
is to design new loss functions [77], [170] by reshaping the
standard cross-entropy loss to put more focus on difficult,
misclassified examples. The recent A-Fast-RCNN detection
model [164], which utilizes GANs to handle occlusion and
deformation samples, is also regarded as a hard miningapproach
example. Pang et al. [172] proposed an IoUbalanced
sample method to adaptively select high-quality
negative examples in the proposal candidates for stabilizing
the training process.
In Earth observation literature, recent research works reveal
that detection data sets contain an overwhelming number
of easy examples and only a few difficult examples. Many
scholars have therefore tried to mine the more representative
difficult examples to balance the proportion of foreground-
background class examples. Traditional methods usually
freeze the model to mine negative examples; however, positive
sample mining is also essential to avoid missed detection.
Besides, freezing the model to collect difficult examples
would dramatically slow the progress of the model.
Cheng et al. [173] developed a two-step iterative training
strategy, which alternates between updating the detection
model given to the training set and adaptively
selecting the difficult negative examples for updating
the detection model. Focusing on airport detection, Cai
et al. [174] and Xu et al. [175] applied cascade strategies
to automatically select difficult examples according to
the loss values of proposals. The cascade strategies significantly
inhibited the false alarms that existed in airport
detection.
DECEMBER 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
HANDLING INSUFFICIENT TRAINING DATA
The difficulty of acquiring annotation samples means that
the training data are not usually sufficient for obtaining
ideal models, and data augmentation is the most straightforward
method for increasing training data. In addition,
research scholars have developed many methods to address
the problem; these can be divided into three categories:
transfer learning (TL), active learning (AL), and weak supervised
learning (WSL). TL can effectively transfer welltrained
knowledge from one or more source tasks to another
task; this needs only a small amount of labeled data
and eliminates the drudgery of preliminary learning [176]-
[179]. Dong et al. [180] proposed a Sig-NMS-based faster
RCNN with TL; this can annotate not only the class of an
object but also its location. Chan-Hon-Tong et al. [181] and
Kellenberger et al. [182] exploited an AL-based strategy to
find very confident samples for the quick retrieval of TPs in
the target data set.
Another method, WSL, addresses the data insufficiency
problem by training detection using image-level labels
only. Recently, research works on WSL have followed different
branches. Some scholars have utilized multi-instance
learning for WSL [183]-[185]. If an image contains many
object candidates, it is considered to involve a set of labeled
bags, with each bag containing many instances; image-level
annotation acts as the label. The object detector is then obtained
by alternating detector training, using the detector
to select the most likely object instances in positive images.
Research works on CNN visualization have demonstrated
that the convolution layer of a CNN model behaves as a
target detector even though there is no supervision of the
object's location. Therefore, class-activation mapping sheds
light on a way to give a CNN model localization ability by
training it on image-level labels [186]-[188]. Some scholars
automatically select the most informative regions and train
them with image-level annotation [189]. Another method
masks out different regions of the image to localize the object
[190]. Interactive annotation [184] and generative adversarial
training have also been used for WSL [191].
To address the problem of a lack of annotated HRRS
data sets, Zhang et al. [192] employed an iterative, weakly
supervised learning framework to automatically mine and
augment a training data set from the original images.
Cao et al. [193] proposed a novel multi-instance-detection
algorithm based on learning, using it to learn instancewise
detectors from such a " weak annotation. " In the algorithm,
a density estimator is adopted to estimate the density map
of vehicle instances from the positive regions; a multi-instance
SVM is then trained to classify and locate vehicle
instances from this map. Although existing WSL methods
take scenes as being isolated and ignore the mutual cues
between scene pairs when optimizing deep networks,
Li et al. [194] exploited both the separate scene category
information and the mutual cues between scene pairs to
train deep networks well enough to pursue superior objectdetection
performance.
25
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
