IEEE Robotics & Automation Magazine - September 2018 - 59

in a considerably larger behavior tree at runtime once Nao
detects a second ball. We developed this tree by first creating
and debugging simple trees, then reusing these as leaf nodes
of other simple trees, which we also debugged, and so forth
until we obtained the desired complex behavior.
The tree has Nao sharing attention between the two balls.
The robot uses virtual targets to which to walk for either
aligning itself with the two balls to kick one in the direction of
the other or for walking toward one while avoiding the other
and performing a reactive arm-tracking behavior based on
inverse kinematics.

Figure 8. Apollo executing an interactive manipulation behavior.

Interactive Manipulation Task Executed on Apollo
Software Stack
Apollo is a fixed-base manipulation platform equipped with
seven-degrees of freedom Kuka LWR IV arms, three-fingered
Barrett Hands, and a red, green, blue depth camera (Asus
Xtion) mounted on an active humanoid head (Sarcos). Its
operating system, simulation laboratory (SL), runs over a realtime-patched Linux kernel (Xenomai), and performs torque
control using an inverse dynamics controller to track the
desired joint state [12]. The controllers implemented in SL are
designed to remain stable in the context of rapid behavior
switching, which is a requirement for safe interfacing with
Playful. The ROS is used for broadcasting RGB images and
point clouds collected from the head-mounted camera, applying sensor processing using the point cloud library and the
OpenCV libraries, and calculating transformations between
various reference frames. ROS nodes broadcast sensory information, transforms, and joint states. This approach corresponds to standard robotics practices.
The Playful tree is added above the ROS. The sensory
leaf nodes subscribe to the underlying topics, and the motor
leaf nodes publish the desired joint states, which are then
transmitted to the underlying SL controller via the ROS in
real time.
Original Behavior
We used Playful to develop a human-robot interaction
application. We controlled the right arm of the robot to mirror the motion of the arm of the closest person standing in
front of it. When presented with a cup, the robot grasps it
and places it at the location indicated by the person. As the
cup moves, new grasping postures are dynamically recomputed. Both for the sake of keeping objects centered in the
field of vision and giving an impression of liveliness, the
robot alternates its gaze between the presented cup, the person, and the person's hand.
A picture of Apollo performing the application is shown in
Figure 8, and the behavior is also shown in the support video
[16]. As can be seen, the robot is highly interactive. However,
we effectively managed the complexity with reusable components of modest size. All of the code and modules used are
based on well-known accessible libraries, part of the roboticist's standard tool kit. We were able to achieve behavioral

richness and reactivity because of Playful's efficient code organization and expressiveness.
Extended Behavior
We exclusively extended the behavior by reusing existing
nodes and evaluations. This extension has the robot dropping
the cup it has in hand if
any other cup is presented
to it. To obtain the desired
We were able to achieve
results, the nodes for arm
tracking, motion planbehavioral richness
ning, and evaluations that
assess the relative posiand reactivity because
tions of objects were applied in a new branch.
of Playful's efficient
This extension required
the addition of only six
code organization
lines to the Playful script
and did not require deand expressiveness.
velopment of any new
Python component. Yet it
extends the behavior considerably. This extended behavior
can be found in the support video [16]. A video of Apollo
running this application when interacting with various guests
is shown online [18].
Future Work
Playful is suitable for the creation of first-order reactive
behavior. Our efforts are now directed toward evaluating
Playful as an interface between higher-level decision-making
systems and reactive execution. A standard approach consists
of using planners to generate sequences of tasks to execute
and monitor. Our platform is different.
In Playful, all of the logic is implemented via evaluations.
As evaluations correspond to arbitrary Python code, they
may be used as bridges with any higher-level software that
supports interfacing with Python. Via evaluations, other
software may thus command activation or deactivation of
subtrees, modify priorities, or change the state of the running program.
To evaluate this approach, we have interfaced Playful
with a case-supported principle-based behavior paradigm, a
september 2018

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - September 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2018