IEEE Robotics & Automation Magazine - December 2015 - 152

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Figure 2. The class evolution. Images for (a), (d), and (g) classes
corridor, (b), (e), and (h) elevator area, and (c), (f), and (i)
student office.

Participant Proposals and Remarkable Results
First Edition
Twenty-nine different groups registered for the first edition
of the Robot Vision challenge [16], organized in 2009.
Among them, seven groups submitted at least one run to the
competition, with four groups competing both the mandatory and the optional task. For the mandatory task, a wide

Results

First

Second

Third

Fourth

Maxmimum score

1,690

5,102

2,741

2,445

793

3,824

677

2,071

Second Edition
The 2010@ICPR edition [32] had participation similar to the
first edition, with eight participating groups, out of which four
competed for both the mandatory and the optional task. As
mentioned above, participants were provided with the stereo
images obtained from two cameras mounted on the robot in
this edition. Among the proposals for the mandatory task, the
approach adopted by the computer vision and geometry (CVG) group [33] stood out for its full usage of the stereo
images to reconstruct the 3-D geometry of the rooms. Using
the reconstructed geometry, the authors could extract viewpoint invariant features [34], while canonical SIFT features
were also extracted from monocular images.
Using this approach, the CVG group achieved
the best score of 3,824 points out of 5,102. The
optional task was once again won by the SIMD
group [35], this time by computing SIFT similarFifth
ities between test frames and a set of training
7,030
candidate frames, which was selected by means
6,033
of clustering techniques.

Score second group

787

3,674

662

1,817

5,722

Score third group

784

3,372.5

638

1,348

5,004.750

Score fourth group

544

3,344

253

1,225

4,638.250

Score fifth group

511

3,293

62

1,028

4,497.875

Score sixth group

456

3,272

-20

551

Score seventh group

-

2,922.5

-77

462

-

Score eighth group

-

2,283.5

-

-70

-

Table 4. The participation.
Task Edition
Participation

First Second Third Fourth Fifth

Registered groups

19

28

71

43

39

Participant groups

7

8

7

8

6

Working notes
submitted

5

3

3

4

2

Table 5. The mandatory task results.
Task Edition

Score first group

152

*

range of techniques was proposed for the image representation and classification steps. The best result, 793 points out of
1,690, was obtained by the Idiap group using a multicue discriminative approach [25]. The visual cues considered by this
group included two global descriptors: 1) composite receptive field histogram (CRFH) [26] and 2) principal component analysis of census transform [27]; as well as two local
descriptors: 1) scale invariant feature transform (SIFT) [27]
and 2) speeded-up robust features (SURF) [28]. A support
vector machine (SVM) [29] was trained for each visual cue,
and a high-level cue integration scheme, the discriminative
accumulation scheme (DAS) [30], was used to combine the
scores provided by the different SVMs. Interestingly, the
DAS cue-integration method assigned the highest weight to
the SIFT features and a zero weight to the SURF features (i.e.,
the SURF features were actually not used in the final system).
A threshold-based system was then used to refrain from
making a decision. For the optional task, the best result,
916.5 points, was obtained by the Sistemas Inteligentes y
Mineria de Datos (SIMD) group [31] using a particle filter
approach to estimate the position of the robot given the previous position. A set of candidate positions (particles) were
sampled around the previous estimated position and separately evaluated using the similarity between the query image
and the training images whose annotated positions (obtained
using the annotated odometry information) were the closest
to the considered particle.

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

DECEMBER 2015

-487

Third Edition
The third edition of the Robot Vision challenge
[36] was attended by seven groups, three of
which particpated in both the mandatory and
optional tasks. For this edition, participants were
asked to classify images recorded on a floor different from the one used to acquire the training
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2015
