IEEE Robotics & Automation Magazine - September 2018 - 65

system from some initial state or scene to a desired state or
scene. This planning process is structured as an alternation
between task planning to identify the discrete actions and
motion planning to identify the paths for each action. Some
task plans may include infeasible actions, e.g., picking up an
object that is blocked by something in front of it. In this
case, motion planning would fail, i.e., exceed a timeout, and
we would go back to the task planner to find a different
task plan, e.g., first moving the blocking object out of the
way. The "TMP" section discusses our implementation of a
TM planner.
TM Control
The TM control phase executes the plan in real time. Each
path produced by the motion planner is a sequence of waypoints the robot must move through. To execute this motion
plan, we compute a reference position, velocity, and so forth
for the robot at each time step by interpolating between the
waypoints. In addition, we must correct the positioning error

in following the motion plan through feedback control. Finally, we operate the gripper to grasp and release objects as specified by the actions in the plan. The "Output and Execution"
section discusses our control and execution implementation.
TMKit Implementation
Our TM system, TMKit, may interest researchers looking to
use TMKit for the algorithm of [1] and [2] or for implementing new TMP approaches. Figure 4 outlines the major software components in our system implementation. TMP
involves many different software modules, and our design
choices were also influenced by the need to support real-time
execution. The key to integrating these components in our
system was identifying the appropriate abstractions for the
task and motion domains and relating these abstractions
through the domain semantics. Using these suitable abstractions not only eases development but also increases flexibility
by providing a uniform interface to domain information
such as task state or scene geometry.

BLAS/
LAPACK

Linear
Algebra

SMTlib

Inverse
Kinematics

Control

SMT

Cartesian
Python
Incremental
Task Planner

TASK
LANGUAGE

DOMAIN
SEMANTICS

Binary

Key

Module

PDDL

URDF
Scene
File

Motion
Planning
OMPL

Visualization

SCENE
COMPILER

Collision
Checking
SDL/
OpenGL

FCL

POV-Ray

Wavefront

File/Input/Output
External Tool

MOTION
SCENE GRAPH

C Compiler

PDDL Parser

Data Structure

Lisp

Blender
STL

COLLADA

Uses
Input/Output

Figure 4. A map of software components. The key data structures are the task language and the motion scene graph. These data
structures are connected by the domain semantics definitions. The scene compiler is also an important component. This system
integrates the following external tools and formats: a basic linear algebra subprograms/linear algebra package (BLAS/LAPACK),
high-performance linear algebra routines with many vendor-optimized implementations; collaborative design activity (COLLADA), an
interchange file format for 3-D applications; a flexible collision library (FCL), a popular software library for collision checking; a GNU
compiler collection (GCC), a compiler suite from the GNU project; the OMPL, a popular software library for sampling-based motion
planning; Persistence of Vision Raytracer (POV-Ray), an open-source ray-tracing program; a planning domain definition language
(PDDL), a cross-platform library to access graphics, audio, mouse, keyboard, and so forth; the Simple DirectMedia Layer (SDL), a
cross-platform library to access graphics, audio, mouse, keyboard, and such; SMT, a decision-problem-combining logic and additional
theories, e.g., integer constraints, lists, and arrays; stereolithography (STL), a file format for computer-assisted design software;
universal robot definition format (URDF), an XML file type for robot kinematics; XML, a tree-structured, general-purpose file format;
and Z3, a high-performance theorem prover/SMT solver.

september 2018

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - September 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2018