IEEE Robotics & Automation Magazine - December 2019 - 114

scalability. All of our recent humanoid robots [2], [23] share
the same cognitive and functional software architecture
implemented in ArmarX [24], [41]. As a complex humanoid
robot system, ArmarX is not only a robot middleware but
also a complete functional cognitive architecture offering
ready-to-use modules on multiple levels of abstraction. Figure 5 shows the key modules of the architecture, which are
divided into three levels of abstraction: 1) real-time control
and hardware abstraction, 2) perception and the robot memory system, and 3) task planning.
On the lowest level, nearly all data are subsymbolic. The
hardware abstraction layer connects with the robot-agnostic,
real-time control framework. This abstraction level provides
its data to the upper levels and receives commands from the
highest level. On the middle level, perception modules for sensor processing such as object localization, self-localization, and
human pose estimation are located. Their results are stored in
the ArmarX memory system, MemoryX, which includes
modules such as the working memory for a consistent world
state, long-term memory for self-experienced data, and prior
knowledge that contains data provided by robot programmers
(e.g., 3D object and world models) as well as preprogrammed
trajectories. The result of the sensor processing is usually

High Level

Task Controller

represented in (but not restricted to) easy-to-process formats
such as 6D poses or symbolic relations. On the highest
abstraction level, reasoning modules, such as natural language
understanding, symbolic planning, and statecharts, coordinate
the robot subsystems primarily on a symbolic level.
Figure 6 displays an abstract depiction of an exemplary
statechart (in this case, for the handover task described in the
"Human Activity Recognition and Active Handover" section)
and the interconnections between respective perception and
the control modules. All of the modules run in parallel and
provide their data asynchronously to the statecharts, which
generate targets for the controllers that run synchronously in
the real-time layer.
One key principle of the robot architecture is the modularity of the software based on common interfaces. This
facilitates the easy exchange or insertion of new software
modules to be used on multiple robot platforms, which is
especially important for the purpose of research. Due to
this modularity, failures of individual modules do not affect
the complete system. The complexity of the robot's software
system results in a high number of interconnected modules
that create a complicated network of dependencies. To provide an assessment of the system's state, ArmarX presents

Speech Commands and RoNoH Based on NLU

Natural Language
Understanding (NLU)

Scene
Understanding

Symbolic
Planning

RoNoH Based
on Human Pose

Task Coordination
(Statecharts)

Task Triggering

World State

Robot Memory

Middle Level

Prior Knowledge

Long-Term Memory

Key
RoNoH Control
and Data Flow

General Control
and Data Flow

Working Memory
Controller
Parametrization

Perception
Scene Registration
and Self-Localization

Object Localization

Human Pose Estimation

Images and
Point Clouds

Low Level

Real-Time Control Framework
Cartesian Velocity Cartesian Impedance
Controller
Controller

Trajectory
Controller

CCDMP Controller

Torque Controller

Position
Controller

Compliant Velocity
Controller

Velocity Controller

RoNoH Based
on Haptics

Hardware Abstraction Layer

Figure 5. An overview of ArmarX, the functional software architecture of ARMAR-6. The image is divided into its three main conceptual
levels and their principal interconnections. The information is increasingly abstracted from low (e.g., raw sensor values) to high levels
(e.g., symbolic predicates). RoNoH: recognition of the need of help; CCDMP: Coordinate Change Dynamic Movement Primitive.

114

IEEE ROBOTICS & AUTOMATION MAGAZINE

DECEMBER 2019

IEEE Robotics & Automation Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2019