IEEE Signal Processing Magazine - January 2018 - 95

rows in Table 4 show, when using the same refinements steps, the
deeper backbone model can obtain the better final performance.
However, as the last refinement step plays a very importance
role in boosting the final performance, using the ResNet as the
backbone model cannot outperform the usage of the VGG net.
As the network architectures similar to DHS are widely used in
other modern detection approaches, the aforementioned analysis
is important for us to balance the tradeoff brought by the ResNet.

Experimental comparison of COD methods
In this section, we select several representative CNN-based object
detection methods that were published in the last three years
(2015-2017) for performance comparison on the PASCAL VOC
2007 data set [79], PASCAL VOC 2012 data set [16], and MS
COCO data set [80]. These methods include fast R-CNN [18],
faster R-CNN [40], Inside-Outside Net (ION) [63], G-CNN [59],
SSD [58], YOLO [57], and TOLOv2 [60]. Tables 5-7 report the
detection results on PASCAL VOC 2007, PASCAL VOC 2012,
and COCO data sets, respectively, by using the representative state-of-the-art methods. Table 8 presents the accuracy
and speed comparison of various methods on the PASCAL
VOC 2007.

As can be seen from Tables 5-7, regression-based COD
methods, such as SSD [58] and TOLOv2 [60], could obtain
comparatively even better accuracy compared with object
proposal-based methods, such as fast R-CNN [18] and faster
R-CNN [40]. Mainly, SSD512 [58] achieved the best accuracy on all three benchmarks (PASCAL VOC 2007, PASCAL VOC 2012, and COCO data sets). The computational
cost comparison shown in Table 8 suggests that regressionbased COD methods are much faster than object proposalbased methods with a comparable accuracy. Among them,
YOLOv2 is the most competitive real-time detector and can
also run at different resolutions for an easy tradeoff between
speed and accuracy. The comprehensive comparison results
in Tables 5-8 show that a regression-based method is a promising COD direction.
Recently, most detection frameworks, such as fast R-CNN
[18], faster R-CNN [40], and SSD [58], rely on VGGNet-16
[25] as the backbone CNN model. Although VGGNet-16 is
a powerful, accurate classification network, it is computationally expensive. The convolutional layers of VGGNet-16
require 30.69 billion floating point operations for a single
pass over an image with a size of 224 × 224 pixels. The more

Table 5. Object detection results on the PASCAL VOC 2007 test set. The VGGNet-16 is the default CNN model for all methods
if not following instructions. All methods (except for G-CNN) are trained on a union of VOC2007 trainval and VOC2012 trainval.
The G-CNN method is trained on VOC2007 trainval set.
Method

aero bike bird boat bott bus

car

cat

chair cow table dog

horse mbik pson plant sheep sofa

train tv

mAP

Fast R-CNN

77.0 78.1 69.3 59.4 38.3 81.6 78.6 86.7 42.8 78.8 68.9 84.7 82.0 76.6 69.9 31.8 70.1 74.8 80.4 70.4 70.0

Faster R-CNN 76.5 79.0 70.9 65.5 52.1 83.1 84.7 86.4 52.0 81.9 65.7 84.8 84.6 77.5 76.7 38.8 73.6 73.9 83.0 72.6 73.2
Faster R-CNN 79.8 80.7 76.2 68.3 55.9 85.1 85.3 89.8 56.7 87.8 69.4 88.3 88.9 80.9 78.4 41.7 78.6 79.8 85.3 72.0 76.4
(ResNet-101)
ION

78.2 79.1 76.8 61.5 54.7 81.9 84.3 88.3 53.1 78.3 71.6 85.9 84.8 81.6 74.3 45.6 75.3 72.1 82.6 81.4 74.6

G-CNN

68.3 77.3 68.5 52.4 38.6 78.5 79.5 81.0 47.1 73.6 64.5 77.2 80.5 75.8 66.6 34.3 65.2 64.4 75.6 66.4 66.8

SSD300

75.5 80.2 72.3 66.3 47.6 83.0 84.2 86.1 54.7 78.3 73.9 84.5 85.3 82.6 76.2 48.6 73.9 76.0 83.4 74.0 74.3

SSD512

82.4 84.7 78.4 73.8 53.2 86.2 87.5 86.0 57.8 83.1 70.2 84.9 85.2 83.9 79.7 50.3 77.9 73.9 82.5 75.3 76.8

Table 6. Object detection results on the PASCAL VOC 2012 test set. The VGGNet-16 is the default CNN model for all methods
(except for YOLO and YOLOv2) if not following instructions. All methods (except for ION and G-CNN) are trained on a union of VOC2007
trainval, VOC2007 test, and VOC2012 trainval. ION and G-CNN are trained on a union of VOC2007 trainval and VOC2012 trainval.
Method

aero bike bird boat bott

Fast R-CNN

82.3 78.4 70.8 52.3 38.7 77.8 71.6 89.3 44.2 73.0 55.0 87.5 80.5 80.8 72.0 35.1 68.3 65.7 80.4 64.2 68.4

bus

car

cat

chair cow table dog horse mbik pson plant sheep sofa train tv

mAP

Faster R-CNN 84.9 79.8 74.3 53.9 49.8 77.5 75.9 88.5 45.6 77.1 55.3 86.9 81.7 80.9 79.6 40.1 72.6 60.9 81.2 61.5 70.4
Faster R-CNN 86.5 81.6 77.2 58.0 51.0 78.6 76.6 93.2 48.6 80.4 59.0 92.1 85.3 84.8 80.7 48.1 77.3 66.5 84.7 65.6 73.8
(ResNet-101)
ION

86.9 84.5 75.2 58.2 57.7 80.5 78.3 90.4 54.4 79.9 60.5 88.4 83.0 83.0 81.2 50.7 77.3 67.6 83.5 72.3 74.7

G-CNN

82.0 76.1 69.3 49.9 40.1 75.2 69.5 86.3 42.3 72.3 50.8 84.7 77.8 77.2 68.0 38.1 68.4 59.8 79.1 61.9 66.4

SSD300

85.6 80.1 70.5 57.6 46.2 79.4 76.1 89.2 53.0 77.0 60.8 87.0 83.1 82.3 79.4 45.9 75.9 69.5 81.9 67.5 72.4

SSD512

87.4 82.3 75.8 59.0 52.6 81.7 81.5 90.0 55.4 79.0 59.8 88.4 84.3 84.7 83.3 50.2 78.0 66.3 86.3 72.0 74.9

YOLO

77.0 67.2 57.7 38.3 22.7 68.3 55.9 81.4 36.2 60.8 48.5 77.2 72.3 71.3 63.5 28.9 52.2 54.8 73.9 50.8 57.9

YOLOv2

86.3 82.0 74.8 59.2 51.8 79.8 76.5 90.6 52.1 78.2 58.5 89.3 82.5 83.4 81.3 49.1 77.2 62.4 83.8 68.7 73.4

IEEE SIGNAL PROCESSING MAGAZINE

|

January 2018

|

95
Table of Contents for the Digital Edition of IEEE Signal Processing Magazine - January 2018
