IEEE Robotics & Automation Magazine - June 2011 - 24

Using larger regions (instead of points) as basic units for
learning offers many advantages. As was already shown in
[1], the knowledge about different functional units of objects
can contribute significantly to a correct detection. Moreover, we use training examples that are similar, but not
necessarily the same as the
* objects encountered durOur classification of objects ing the operation of the
robot, thus improving the
generalization of the final
is based on the detection
classifier. In the experimental section, we evalof the different parts that
uate our method on real
3-D scans of indoor scenes
compose them.
* and present our insights
on what would be required from a WWW for robots to support the generalization
of this approach.
Related Work
Although appearance-based object identification works
reasonably well using a variety of techniques, the robustness and scalability of many perception systems remains
an open issue, as identified by Kragic and Vincze [2].
Ideally, a robot should be able to recognize thousands of
objects in a large variety of situations and additionally
detect their poses. We will review some of the steps taken
in this direction and contrast them to the method proposed in this article.
A widely used technique to recognize objects in point
clouds involves local descriptors around individual points.
For example, the spin image descriptor [3] is used by Triebel et al. [4] to recognize objects in laser scans, and it is also
used by de Alarcon et al. [5] to retrieve 3-D objects in databases. More recently, Steder et al. [6] presented the normal

Web
Models
Real
Environment

World
Model

Figure 1. Using furniture models from the WWW together with
a segmented scan of their real environment, the robots create a
world model. See the color legend in Figure 3 (different parts
are indicated by random colors).

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2011

aligned radial features (NARF) descriptor that is well-suited
for detecting objects in range images. Other works apply
relational learning to infer the possible classification of
each individual point by collecting information from
neighboring points. In this sense, Angelov et al. [7] introduce associative Markov networks to segment and classify
3-D points in laser scan data. This method is also applied
by Triebel et al. [8] to recognize objects in indoor environments. All previous methods use individual 3-D
points as primitives for the classification, whereas we use
complete 3-D segments or parts represented by feature
vectors. We believe that parts are more expressive when
explaining objects.
For the detection of complete objects, for example, 3-D
shape contexts and harmonic shape contexts, descriptors
are presented in the work by Frome et al. [9] to recognize
cars in 3-D range scans. In the work by Wu et al. [10],
shape maps are used for 3-D face recognition. Haar features are used in depth and reflectance images to train a
classifier in the work by N€
uchter et al. [11]. In these works,
objects are detected as a whole, whereas we are able to
detect objects by locating only some of their parts, which
results in better detections under occlusions and when
using different viewpoints. Our work shares several ideas
with the approach by Klasing [12], which also detects
objects using a vocabulary of segmented parts. However,
we apply the classifier directly to the point cloud without
looking for isolated objects first.
Part-based object classification in 3-D point clouds has
also been addressed by Huber et al. [13], using point clouds
partitioned by hand. In contrast, we partition the objects in
an unsupervised manner. Ruiz-Correa et al. [14] introduce
an abstract representation of shape classes that encode the
spatial relationships between object parts. The method
applies point signatures, whereas we use descriptors for
complete segments.
Many of the techniques in our approach come from the
vision community. The creation of a vocabulary is based
on the work by Agarwal and Roth [15], and its extension
with a probabilistic Hough voting approach is taken from
Leibe et al. [16]. Voting is also used by Sun et al. [17] to
detect objects by relating image patches to depth information. Geometrical information allows us to have a single
3-D CAD model of an example object in the WWW database, since the different views can be generated by the
robot. Combinations of 3-D and two-dimensional (2-D)
features for part-based detection would definitely improve
the results [18].
For matching problems, random sample consensus
(RANSAC) and its variations are widely used due to the
flexibility and robustness of the algorithm [19], [20]. To
register different views of an object, local tensors are
applied by Mian et al. [21]. Moreover, Rusu et al. [22] limit
the point correspondences by using local features.
Finally, using synthetic data for training data is an
idea that appears in several works [23], [24]. Lai and Fox

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2011