IEEE Robotics & Automation Magazine - December 2015 - 153

Table 6. The optional task results.
Task Edition
Results

First

Second

Maxmimum score

1,690

Third

Fourth

5,102

2,741

4,079

Score first group

916.5 3,881

2,052

3,930

Score second group

890.5 3,783.5

3,859

Score third group

884.5 3,453.5

-67

3,169

Score fourth group

853

1,041

2,822

% Over Maximum Score

100
80
60
40
20
0

Fourth Edition
As mentioned above, the 2012 edition [19] introduced range
images obtained with a Microsoft Kinect sensor. Up to 43
groups were recorded in the task, but only eight submitted
their results. The organizers proposed a baseline method for
both the feature extraction and the classification steps [42].
Regarding feature extraction, the pyramid PHOG [40] (for
visual images) and the normal aligned radial feature [43]
(for depth images) were proposed. Concerning classification
and cue integration, the Batch Strongly Convex Multi Online Kernel Learning (OBSCURE) [44] method was selected. Because of this proposal, the results of the baseline
submission were 462 out of 2,445 for the mandatory task
and 1,041 out of 4,079 for the optional task. The group from
the Universidad Tecnologica Nacional, Cordoba, Argentina
(CIII UTN FRC) [45] was the winner for both tasks with a
score of 2,071 and 3,930, respectively. This group made use
of the depth information (in fact, they were only group that
used it). For low-level features, they used the SIFT descriptor [46], reducing the dimensionality of this descriptor with

First

Second

Third
Edition

Fourth

Fifth

Figure 3. The results out of the maximum score for the
mandatory task.

100
% Over Maximum Score

images. Consequently, this edition of the competition required the algorithms to show higher generalization capabilities. For the second time, the CVG group won the
mandatory task with an approach combining a weighted kNN search using global features, with a geometric verification step [37]. Surprisingly, this winning approach made use
of only a single holistic image representation (GIST) [38].
Furthermore, the GIST representation of a query image was
directly matched with the GIST representations of the training images, by sorting the training images according to their
R 2 distance to the query image. An image was finally assigned to the class of the matching image at the lowest angular distance. This approach obtained a score of 677 points out
of 2,741. For the optional task, the approach proposed by the
Idiap group [39] proved to be the most effective. The proposed multi-cue system combined up to three different visual descriptors (namely, pyramid histogram of oriented
gradients (PHOG) [40], CRFH [26], and principal local binary pattern [41]), in a discriminative multiple kernel SVM.
A door detector was implemented to determine the transition from one room to another, while a stability estimation
algorithm was used to evaluate the stability of the classification process. Using this approach, the Idiap group obtained
the winning score of 2,052 points out of 2,741.

80
60
40
20
0
First

Second
Third
Edition

Fourth

Figure 4. The results out of the maximum score for the optional task.

principal components analysis (PCA) and encoding the information in Fisher Vectors image signatures [47] for both
color and depth images. Finally, the classification phase relied on SVMs. The group
from the Alexandru Ioan
The Robot Vision task
Cuza University, Iasi, Romania (UAIC2012) [48]
has served for sharing
presented an interesting
approach for the mandatechniques and knowledge
tory task, which used a
combination of hue, satubetween worldwide
ration, and value and
RGB color histograms in
researchers.
conjunction with SIFT
descriptors (following a
bag-of-word approach).
They achieved 1,348 points. The group from the Ural Federal University, Yekaterinburg, Russian Federation (USU room
409) [49] proposed the use of a growing Kohonen network.
They obtained 1,225 points in the mandatory task using statistical pixel values like expected value, variance, and standard deviation as inputs.
DECEMBER 2015

IEEE ROBOTICS & AUTOMATION MAGAZINE

153

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2015