IEEE Robotics & Automation Magazine - September 2015 - 98

controllers or steering functions for these systems. Kinodynamic planners in OMPL fall back in such cases on sampling random controls. This makes planning for these
systems extremely challenging; however, if controllers are
available, then OMPL can use them. With a few lines of
code, the command line tool can be modified to allow new
planning algorithms or new types of planning problems to
be specified in the configuration files.
The benchmark configuration files can be created with the
graphical user interface (GUI) included with OMPL.app. A
user can load meshes in a large variety of formats, define start
and goal states, try to solve the problem with different planners, and save the configuration file. The user can also visualize the tree/graph produced by a planning algorithm to get a
sense of the difficulty of a particular problem. In the configuration file, the user can specify whether the solution paths (all
or just the best one) should be saved during benchmarking.
Saved paths can be played back with the GUI.
When defining motion planning problems in code, many
of the limitations of the command line tool go away. Arbitrary state spaces and kinodynamic systems can be used and
different notions of state validity and different optimization
objectives can be defined. In addition, any user-defined planning algorithm can be used. The OMPL application programmer interface (API) imposes only minimal
requirements on new planning algorithms. In particular, the
API is not limited to sampling-based algorithms (in [8], for example, several nonsampling-based planners are integrated
Runs
into OMPL). The low barrier to entry has
P
id INTEGER
lead to numerous contributions of planexperimentid INTEGER
F
ning algorithms from other groups: OMPL
1.0 includes 29 planning algorithms. Since
plannerid INTEGER
F
all these algorithms use the same low-level
graph_states INTEGER
functionality for, e.g., collision checking,
time REAL
benchmarking highlights the differences in
solution_length REAL
the motion planning algorithms themsolution_clearance REAL
selves.
status ENUM
The benchmarking facilities in MoveIt!
[9] are based on and compatible with those
approximate_solution BOOL
in OMPL. The problem setup is somewhat
simplified_solution_length REAL
similar to the OMPL command line tool.
iterations INTEGER
In MoveIt!, robots are specified by Unified
Robot Description Format (URDF) files,
which specify a robot's geometry and kine(Many More Attributes)
matics. Motion planning problems to be
benchmarked are stored in a database.

Defining Motion Planning Problems
The most common benchmark motion planning problems
are those where robots are modeled as rigid bodies due to
their simplicity (it is easy for users to intuitively assess performance). We have developed a simple plain-text file format that describes such problems with a number of
key-value pairs. Robots and environments are specified by
mesh files. The state validity function is, in this case, hardcoded to be a collision checker. Besides the start and goal
positions of the robot, the user can also specify an optimization objective, such as path length, minimum clearance
along the path, or mechanical work. There are several planning algorithms in OMPL that optimize a path with respect
to a specified objective.
(Others that do not supThere is no universal metric port optimization simply
ignore this objective.) It
is also possible to specify
to assess performance of
simple kinodynamic moplanning algorithms across tion planning problems.
OMPL.app, the application layer on top of the
all benchmarks.
core OMPL library, predefines the following systems that can be used: 1) a first-order car, 2) a second-order
car, 3) a blimp, and 4) a quadrotor. We have not developed

Experiments
P
id INTEGER
name VARCHAR(512)
totaltime REAL
timelimit REAL
totaltime REAL
memorylimit REAL
runcount INTEGER
version VARCHAR(128)
hostname VARCHAR(128)
cpuinfo TEXT
date DATETIME
seed INTEGER
setup TEXT

PlannerConfigs
id INTEGER
name VARCHAR(512)
settings TEXT

Progress
runid INTEGER
time REAL
best_cost REAL
iterations INTEGER

F
P

Figure 2. The schema for a database of benchmark results. The P and F denote the
primary and foreign keys of each table, respectively.

IEEE ROBOTICS & AUTOMATION MAGAZINE

September 2015

Specifying Planning Algorithms
Once a motion planning problem has been
specified, the next step is to select one or
more planners that are appropriate for the
given problem. Within OMPL, planners
are divided into two categories: 1) geometric/kinematic planners and 2) kinodynamic planners. The first category can be

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2015