IEEE Robotics & Automation Magazine - December 2019 - 119

sequences for both arms are defined as prior knowledge.
When it is in place, ARMAR-6 verbally confirms that it is
ready to lower the panel. As soon as the technician has fully
unmounted the panel, which the robot again realizes
through haptic feedback, the human-robot team jointly
lowers it. Once lowered, the robot confirms in natural language that it is ready for the next step. To carry the panel to a
temporary location, the technician must step around the
posts of the conveyor setup, which requires him to grasp the
cover panel on one of its corners. ARMAR-6 realizes that the
panel is out of balance and concludes through tactile sensing
that it can help the technician by regrasping it on the opposite corner. It does so and again verbally confirms its readiness for the next step.
The technician then starts to walk toward the intended
placing location of the cover panel. ARMAR-6 has no a priori notion of this location. Instead, it is guided by the movement of the technician, which it senses through the
displacement of its own hands. This force-based method for
guiding is enabled by joint-level compliance in every joint of
the robot's arms, which is one of the principal joint control
modes of ARMAR-6 and has proven to be invaluable for
physical HRI.
Once the cover panel is at the intended location, the technician begins a placing movement. The robot senses the onset
of this motion with its wrist-mounted 6D F/T sensors, recognizes that it can assist the technician with placing the heavy
object, and, in turn, starts the placing motion. Because the
robot's arms are still compliant, the technician can determine
the exact placement position while the robot simply supports
the weight of the panel.
Autonomous Mobile Manipulation
After the cover panel is removed, the technician inspects the
exposed inner workings of the conveyor and decides that
the drive train must be cleaned. Through the use of natural
language, the technician expresses what needs to be done by
saying the drive train needs to be cleaned, as shown in Figure 10(b). ARMAR-6 understands this message using its
ASR system and recognizes that its help with the cleaning
task is needed. Concretely, ARMAR-6 infers that cleaning
requires a cleaning agent and the spray bottle on the nearby
table, which the robot needs to localize and graspe to hand it
to the technician, who has already mounted a ladder to
access the drive train.
To hand over the spray bottle to the human, ARMAR-6
observes the technician, again using 3D human pose tracking.
This action is shown in Figure 9 as a 3D visualization of the
robot's working memory and an overlay skeleton on the input
image of the RGB-D sensor.
Once the technician extends a hand toward the robot,
ARMAR-6 recognizes the activity and initiates the handover.
If the technician withdraws the hand, ARMAR-6 will also
stop the handover and continue observing the human. However, if the technician takes the bottle from the robot's hand,
it senses this with its 6D F/T sensors and opens the hand to

complete the handover action. Once the cleaning task is finished, the technician will hand the spray bottle back to the
robot. Again, the handover intention is recognized, and the
robot's hand is closed as soon as contact is detected. While
the technician stows the ladder, the robot moves back to the
table and returns the spray bottle to its initial location.
Key Aspects
Figure 10 indicates only the primary sensor modality used
in each action. We want to emphasize that most of the perception systems, from laser-based navigation to ASR and 3D
visual perception, are constantly active, providing the robot
with rich information for situational awareness and scene
understanding. Throughout the entire scenario, the robot
recognizes on multiple occasions that the technician needs
its help (illustrated by the red circles in Figure 10). The automatic RoNoH enables the human and robot to work together naturally.
We have executed system validation studies using the
previously mentioned scenario at different locations and
with more than 20 different technicians-most notably
in our lab at the Karlsruhe Institute of Technology, at the
2018 international CEBIT trade fair in front of large
audiences, and at an actual automated fulfillment center
in the United Kingdom. The usability and impact of
ARMAR-6 for the maintenance tasks in this fulfillment
center have been evaluated in a user study using an
accompanying system usability scale questionnaire, conducted in collaboration with the SecondHands project
team members of the École polytechnique fédérale de
Lausanne. In this study, technicians were also asked for
their subjective perception of ARMAR-6 as a coworker
using the Godspeed Questionnaire Series. Details of the
user study are described in [38]. The result of the evaluation helps us to continuously improve on-board algorithms for more intuitive human robot interaction.
A few of ARMAR-6's additional abilities, not showcased
in the aforementioned demonstration scenario, are motion
planning in unknown and dynamic environments, grasping
of unknown objects, and the effortless teach-in of difficult
motions using gravity-compensated, zero-torque control of
the arms.
Conclusions
In this article, we provided a comprehensive overview
of the current capabilities of ARMAR-6, as well as the
vision we are pursuing in our research. We presented the
requirements-driven design and implementation of
ARMAR-6 as a platform for collaborative robotics and the
design choices we made toward this goal. The joint development of hardware and software with the common goal
of creating a robot that "pushes the envelope" for seamless
physical HRI has led to a highly integrated, versatile, and
robust humanoid robot system. The frequent demonstration of ARMAR-6 in its intended scenario, where it helps
a human technician with repairing a conveyor system of
DECEMBER 2019

IEEE ROBOTICS & AUTOMATION MAGAZINE

119

IEEE Robotics & Automation Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2019