IEEE Robotics & Automation Magazine - June 2011 - 75

type-specific subclass of an action such as Translation. Atomic actions, i.e., actions that are not composed
from subactions, represent skills that translate these commands into motions (see the "Architecture: Generic
Components-Action Execution" section for details).
Examples of the information stored in this second database include the action recipe used by robot A in the example detailed in the "RoboEarth: Example Scenario" section,
the location of the bottle, and the number of times the
patient was found in the bed.
Third, services that provide advanced learning and reasoning capabilities at the database level. A first type of service
is illustrated by RoboEarth's reasoning server. It is based on
KnowRob [40] and uses semantic information stored in the
database to perform logical inference. This allows to determine if robot B in the example detailed in the "RoboEarth:
Example Scenario" section meets the requirements needed
to execute the action recipe used by robot A, or if it has the
sensors needed to use the same object-recognition model.
Services may also solely operate on the database.
RoboEarth's learning and reasoning service uses reasoning
techniques [16], [40] to analyze the knowledge saved in the
RoboEarth database and automatically generates new action
recipes and updates prior information. For example, given
multiple task executions, the database can compute probabilities for finding a bottle on top of the cupboard or on the
patient's nightstand. Using the additional information that
cups are likely to be found next to bottles, the service can
automatically create a hypothesis for the probability of finding cups on top of the cupboard. Such cross correlations
between objects can provide powerful priors for object
recognition and help to guide a robot's actions. Additionally, if there are two action recipes that reach the same goal
in different ways, the learning and reasoning service can
detect this, fuse the recipes, and explicitly represent both
alternatives. For example, if robot A was equipped with a
dexterous manipulator but robot B only with a tray, the
component could create a single action recipe "serve drink
to patient" with two branches depending on the robot's abilities, which would have different requirements: the first
branch would require a graspable bottle, whereas the second
branch would require the availability of human or robotic
help to place the bottle on the tray.
Interfaces
The RoboEarth database supports three types of interfaces.
First, a Web interface that allows humans to exchange
information with the database using hypertext mark-up
language (HTML) forms. This interface may be used by a
human to access the RoboEarth database and provide
labels for robot A's perceptions of the bottle.
A second interface is based on the representational state
transfer (REST) architecture [46]. As a stateless system, the
REST interface transfers all necessary information as part of
the request without the need to store any information on the
server. This allows high scalability and fast response times,

because it allows for a number of server-side optimizations
(e.g., clusters with load balancing and fail over capabilities).
The REST-style interface allows to encode data directly into
JavaScript object notation (JSON) [47], which makes it suitable for interaction between robots and the database. Based on
hypertext transfer protocol (HTTP) requests, this interface is
easily implemented in all common programming languages
and supported by most of today's robot platforms. Robot B's
initial request for prior information about the hospital room
as well as RoboEarth's response would both use REST.
Both the Web interface and the REST interface rely on
the request-response message exchange pattern, which
can result in unnecessary traffic because of periodic
requests for updates. To avoid this kind of traffic, a third
interface is based on the publish/subscribe message
exchange pattern, which acts on changes to the database.
This type of interface would allow robot B to subscribe to a
notification for the patient leaving the hospital room to
carry out a room-cleaning operation without constantly
querying the database for the patient's location.
Generic Components
Action Execution
The execution of an action recipe (Figure 5) involves 1)
determining and checking the availability of a set of skills
required for the task, 2) finding an ordering among them that
satisfies the ordering constraints, 3) linking a robot's perceptions to the abstract task description given by the action
recipe, and 4) reliable action execution. The first and second
steps can be resolved by RoboEarth's reasoning server (see
the "Architecture: Database" section for more details). However, the third and fourth steps require a tight integration of
RoboEarth's knowledge representation and the object recognition and action execution system used on the robot.
The action execution component ensures reliable action
execution on the robot by coordinating communication with
the RoboEarth database, monitoring the link between robot
perceptions and actions, and providing failure-handling routines. In case of unresolvable problems, it asks for user input.
For example, if robot B encounters an obstacle while trying
to navigate to the bottle, the action execution component
detects that the action recipe cannot be properly executed
and asks for help. If the issue is solved, the component
ensures a smooth continuation of the action recipe.
Environment Modeling
A world model component combines prior information
available via the RoboEarth database with a robot's sensory
inputs [6]. To guarantee fast and safe performance despite a
potentially unreliable network connection to the RoboEarth
database, it uses a two-level approach. A local world model
supporting high update rates (typically tens of hertz) is executed on the robot. It merges and associates (possibly contradicting) sensor data from the robot's 3-D perception
(e.g., the locations of labeled objects) with tracking data.
JUNE 2011

IEEE ROBOTICS & AUTOMATION MAGAZINE

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2011