IEEE Signal Processing Magazine - January 2018 - 90

However, it is still difficult to detect small-sized objects and
achieve precise localization.
Thereafter, SSD [58] is proposed for improving the YOLO
method. Specifically, it discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios
and scales at each feature map location, sharing the similar idea
with RPN of faster R-CNN. At prediction time, SSD exports the
scores for the presence of each object category in each default
box and generates adjustments to the boxes to better match the
object appearances. In addition, the network combines predictions from multiple feature maps with different resolutions to
handle objects with various sizes. With the introduction of
multiscale feature maps and the default boxes mechanism, SSD
has achieved significant performance improvements to detect
small-sized objects and also improved localization accuracy
when compared with YOLO.
In addition, some recent works have also been developed
to further boost the performance of CNN-based approaches
for COD, such as hard negative mining [61], feature enhancement [41], [62], contextual information fusion [63]-[65], and
so on. For example, to increase the capability to handle challenging situations with object rotation, within-class variability,
and between-class similarity, Cheng et al. [62] proposed a rotation-invariant and Fisher-discriminative CNN model. Building upon the existing high-capacity CNN architectures, it was
implemented by additionally introducing a rotation-invariant
layer and a Fisher-discriminative layer, respectively.

Relationship among OD, SOD, and COD
Although OD, SOD, and COD are three individual research
directions in object detection, a rich relationship can be
observed among them.

Relationship between OD and SOD
On one hand, bottom-up SOD is able to provide informative
prior knowledge to OD. Intuitively, the extracted boundingbox locations that are more attractive to the human visual system (more salient in the image scenes) would be more likely
to contain the objects of interest. Based on this observation,
several OD approaches have been designed by relying on
some saliency cues. For example, one of the most classic OD
approaches [1] selects object proposals by using three saliency
cues, i.e., the multiscale saliency, color contrast, and edge density. Similarly, the work in [66] defined objectness as window
saliency, which is the cost for composing the window by using
the remaining parts of the image. This definition essentially
subsumes the global rarity principle and extends it from pixel
level (for SOD) to window level (for OD). In addition, Cheng
et al. considered OD as a special case of bottom-up SOD [67],
which indicates that OD can be effectively formulated by using
the detection principles of SOD. Erhan et al. [37] also proposed
a saliency-inspired neural network model for OD and achieved
a promising performance.
On the other hand, some bottom-up SOD approaches are
also established upon the OD results. When provided with the
bounding boxes generated by OD, the SOD problem can be
90

simplified as selecting the salient bounding boxes from the
nonsalient ones. Based on this intuition, Chang et al. [68] proposed to integrate the objectness prior (including object size
and location) and saliency prior to detect salient objects via a
unified graphical model. Jiang et al. [69] integrated the objectness prior with focusness and objectness to keep completeness
of detected salient regions for SOD. Li et al. [70] proposed to
treat object candidates with high saliency values obtained from
fixation prediction as salient objects. Since SOD requires uniformly highlighting the complete salient object, which differs
from traditional saliency detection that requires only highlighting distinct local regions, the perceptibility of objects should be
naturally encoded into effective SOD models. Objectness prior
naturally provides an efficacious solution for this requirement.

Relationship between SOD and COD
As top-down SOD is highly task driven and knowledge driven,
it requires high-level understanding of visual scenes, especially the category-level information of objects in the scenes. To
achieve the goal of locating the intended objects in the scene,
top-down SOD approaches usually need to acquire top-down
knowledge for guiding the detection process [51]. Such topdown knowledge might come from memory (i.e., locating
objects in the scene using knowledge from the corresponding
training data, which is the model-based object detection) or
object association (i.e., locating the corresponding objects in
the scene using known or unknown exemplars, which is the
exemplar-based object detection [71]-[73]). For example, in
[50], Yang et al. detected top-down saliency by joint CRF and
dictionary learning. The CRF model was initialed by training a linear SVM on the image patch representation and the
corresponding patch labels, which essentially is a patch-level
category-specific object detector. Some experimental results
or discussions by previous works indicating that top-down
cues provided by some category-specific object detectors (e.g.,
humans, faces, cars, words appearing in a given image, etc.)
are playing an important role in visual attention mechanism
can also be found in previous works [74], [75].
Besides SOD, top-down SOD especially can, in turn, provide helpful category-specific object prior for COD, particularly under weak supervision. As we know, weakly supervised
object detection approaches [3], [76], [77] aim at learning
category-specific object detectors only with image-level tags
rather than the instance-level bounding-box annotations. In
this scenario, how to obtain the initial object locations for
certain object categories is a primary problem that needs to
be addressed. By using the SOD approaches, [3] and [76] initialized category-specific object locations effectively and then
adopted the iterative learning schemes to gradually refine the
object detectors and locations in an iterative fashion. When the
learning process converges, stronger object detectors can be
learned to perform COD in test data. References [52], [53], and
[77] also proposed to apply top-down saliency detection to discover the object locations in the weakly labeled training image,
which subsequently can be used to train the category-specific
object detectors.

IEEE SIGNAL PROCESSING MAGAZINE

January 2018

Table of Contents for the Digital Edition of IEEE Signal Processing Magazine - January 2018