IEEE Robotics & Automation Magazine - June 2012 - 69

Figure 12. The Arm Navigation Wizard window being used to
set up a kinematic chain group for the TUM-Rosie robot.

wiki ). The arm navigation framework includes complete
instructions [33] on installation and running the software
for application on a new robot. The first application
built up using our system was an autonomous pick-andplace demo for tabletop manipulation. The application
is designed to be completely autonomous and has a simple
goal to move objects around on planar support surfaces,
e.g., tables or counters. Two independent actions form the
basis of the application: a pick action to pick up objects
and a place action to place objects on a support surface.
The pick-and-place application uses these two actions to
move all objects on one side of a table to the opposite side
and then switches sides and continues. Figure 13(a) shows
an example of this application in action. As mentioned
earlier, the system works both with known objects, for
which grasps have been precomputed, and with novel
objects, for which new grasps need to be constructed.
In [34], an autonomous beverage fetching application
was designed, based around the motion planning components of our system. The application used a state-machinelike architecture for automated retreival of beverages from
a fridge based on user requests. It also imposed constraints
on the motion plans to keep the beverage containers
upright while they are being manipulated. Figure 13(b)
shows a snapshot from the application in action.

(a)

Our system was further combined with a cart-pushing
application to execute a cleanup application. The cart
allowed the PR2 to carry multiple objects on a cluttered
tabletop through a cluttered office environment. The
PR2 loaded the cart autonomously using our system,
although the choice of objects to pick up and retrieve
was made by a human. Figure 13(c) shows a snapshot from
this application.
Our system has further been used by several external
groups. In [35], our system was used in the PR2 Remote
Laboratory. The laboratory was intended to allow remote
participants to use a PR2 robot. A Web interface that uses
our systems allowed remote users to pick and place objects
on a tabletop, and the application was successful in allowing users to manipulate a wide variety of objects. Our system also served as a testbed for several other researchers
integrating new motion planning algorithms [16], [17] and
is also used in a cookie baking application in [36], where
the PR2 robot manipulated several items in a kitchen tabletop environment.
Lessons Learned
Our experience with the real-world deployment of the
mobile pick-and-place system has clearly showed the
importance of having good situational awareness in
human environments. In particular, we have found a probabilistic representation to be important, especially in dealing with noisy sensors and moving obstacles in the
environment (especially humans). The separation in the
representation of objects from the rest of the environment
has been very helpful, allowing for the maintenance of
object models at the higher resolution required for manipulation (e.g., grasping) while allowing other parts of the
environment to be maintained at a less expensive lower
resolution for motion planning. We have also found that
semantic representations of relevant parts of the environment, e.g., tables or shelves, aid in high-level decisionmaking by abstracting information away from a detailed,
gridlike representation. Finally, maintaining a persistent
model of the environment is important for global

(b)

(c)

Figure 13. Applications of our system: (a) A pick-and-place application for moving objects, (b) a beverage retrieval application, and
(c) the PR2 using a cart to carry objects in a cluttered office environment.

JUNE 2012

IEEE ROBOTICS & AUTOMATION MAGAZINE

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2012