IEEE Robotics & Automation Magazine - December 2019 - 34

motion or an equilibrium point in a few steps is a simple and effective way to guarantee that it is able to avoid
falling. This also provides a simple method to detect the
risk of an imminent fall to trigger fall-mitigation behaviors when appropriate.
The Comanoid consortium adopted a standard multistage framework for walking motion generation and control: 1) a short sequence of step positions and phase
durations is proposed depending on robot current and goal
states as well as the environment (i.e., obstacles and people); 2) the resulting motion of the CoM and contact forces
are computed, making certain that a cyclic motion or an
equilibrium point can be reached within a few steps; and 3)
the robot is controlled to realize the computed CoM
motion and contact forces.
We devised several approaches wvithin this common
framework to improve reliability and safety. Some address
stages 1 and 2 in the previous paragraph separately [11],
while other approaches do so in a single stage [12]. This is
slightly more involved numerically, but it improves the
robot's capacity to react effectively to a dynamic environment. In static environments, stages 1 and 2 can be considered once every few steps. Otherwise, e.g., with
workers moving around, these stages are reevaluated more
often, using approaches based on model predictive control (MPC).
Capturability approaches guarantee that the robot is consistently in a viable state and is always able to stop safely
[12]. Alternative approaches investigate the more general
concept of boundedness [13]. One approach that integrates
safety guidelines w.r.t. surrounding humans is to adapt the
current goal of the robot w.r.t. the current state of the environment [14]. One can also integrate collision mitigation
and passive safety constraints directly in a combination of
stages 1 and 2 [12]. This is more involved numerically, but it
improves the robot's capacity to navigate safely in the presence of workers.
To negotiate uneven ground and stairs, the robot's
CoM height should be adjustable, thus introducing the
nonlinearities present in stage 2. These nonlinearities are
sometimes neglected, but at the risk of failure. We handle
them explicitly by considering a piecewise, linear 3D trajectory of the DCM [11], or we bound them by constraining the height variations of the CoM above the
ground, adapting its capturability and boundedness
accordingly [15].
This diversity of approaches has been used to demonstrate two different humanoid robots capable of navigating
in a typical Airbus environment, walking reliably to their
exact destinations to complete effectively the assigned
manipulation tasks. This requires a tight integration of
walking with visual servoing and SLAM, multicontact
phases, and safety guidelines. This tight integration is arguably the biggest achievement of the project, as walking is
not addressed independently from navigation, manipulation, and safety issues.
34

IEEE ROBOTICS & AUTOMATION MAGAZINE

DECEMBER 2019

In-Site and In-Craft Localization and Mapping
Autonomous SLAM in aircraft manufacturing is a fundamental capability for practical use of humanoid robots in a
real-world setting. Few real-time (RT) approaches have
been proposed that can account for dynamic environments
and long-term incremental changes (e.g., the manufacturing process and lighting variation) while maintaining positioning accuracy or tolerating loss of tracking. Moreover,
assembly operations require continuous correspondence
between the evolving digital mockup and the reality of the
airplane assembly. Consequently, semantic knowledge is
highly necessary.
Image-based keyframe navigation was used for its efficiency and accuracy in humanoid positioning because it allows
for closing the feedback control loop in the sensor space, subsequently avoiding drift, improving robustness, and enabling
loop closure and relocalization (see the results in the "Integration and Experiments" section).
Keyframe SLAM and Image-Based Navigation
A direct, multi-keyframe approach is used to perform redgreen-blue-depth (RGB-D) SLAM for navigation [16]. The
sensor pose p ! se(3) is estimated w.r.t. the set of closest keyframes by minimizing the error between the current frame
and a predicted one. The current measurement vector for
each intensity and depth i is defined as M i = [P i< I i]< ! R 4.
The predicted keyframe M )i ! R 4 is obtained by blending the
n closest keyframes (typically five) at the last pose estimate.
The point-to-hyperplane iterative closest-point approach [17]
is then used to estimate the pose iteratively from the following
error function:
e i (p) = N )i <(M )i - w (M i, p)),

(2)

4
where the normals N )<
i ! R are computed once on the referenced 4D measurement vector and w ($) is the warping
function that transforms the current image to the reference,
based on the current pose estimate.
This basic alignment procedure is extended to large environments by using a keyframe graph, built and refined incrementally as mapping is performed. To take advantage of the
topometric keyframe representation, the target position is
given as a sensor-based keyframe to reach. This permits the
direct image-based error defined in (2) to be minimized by
the robot controller, effectively allowing the robot to position
itself with high accuracy locally while tolerating global drift.

Map Reuse, Place Recognition, and
Time-Varying Environments
Due to the complexity of the manufacturing site's factors, such
as workers, tools, and assembly increments, a prior keyframe
graph was created from a static environment and then reused
online. To ensure robot positioning, pose uncertainty and
sensor-based errors were monitored to determine tracking
loss and trigger relocalization (i.e., pose estimation w.r.t. the
nearest keyframes). In addition, the robot's viewing direction

IEEE Robotics & Automation Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2019