IEEE Robotics & Automation Magazine - June 2020 - 96

length, while also constraining the robot's minimal
and maximal step velocity.
Emerging Strategies and Quality of Fitting
From the x and z CoM position trajectories in Figure 8(a)
and (b), it can be seen that MJMPC generates trajectories
from which the push recovery strategies (Figure 1) emerge
naturally. From the data of the DRL policy, we found that
very little angular momentum was generated (see the "Analyzing the AI Policy" section) and, consequently do not regulate the angular momentum of the humanoid. Typical for the
ankle strategy [blue solid line, Figure 8(a) and (b)], the CoP
is moved within the support polygon to move the CoM position. The CoM height modulation emerges from regulating
the CoM height to its nominal position. For large pushes, the
step optimizer sets a new reference position, and, thus, stepping behavior emerges.
To show the quality of fit, we identify the open parameters by optimizing these via least square error between the
controller and DRL policy. We apply k-fold cross validation across 2,000 trials of the robot and show the mean
and standard deviation in Table 2 with an average coefficient of determination of 0.95 and 0.91 for the x and z
directions, respectively.

Ankle Simulated
Hip Simulated

As presented in Figure 8(a) and (b), using the methodology described in the "Determining Final Time via Data Fitting" section, a final time t f can be chosen such that the
engineered policy fits the DRL, indicating that MJMPC is a
suitable controller for resembling the AI policy. The fit for the
vertical component is slightly worse due to the fact that the
approximation of t f for the CoM height was not as good as it
was for the sagittal component.
Realizability of MJMPC on Real Systems
In this section, we propose a framework for the real-world
deployment of the generated push recovery motions and
demonstrate the feasibility of the motions generated from
MJMPC. To deploy the push recovery motions on a real
robot while guaranteeing stability and implementability, a
whole-body controller is included in the control framework
(Figure 12). To this end, quadratic programming (QP)-based
whole-body controllers can be leveraged to track reference
motions while providing stability; these were successfully
deployed on humanoids such as Valkyrie [6], Atlas [14], and
HRP-2 [15]. Aside from their ability to incorporate stability
and feasibility, thus guaranteeing constraints in the optimization problem formulation, off-the-shelf QP solvers can solve
QP problems extremely quickly and hence enable loop

Ankle Real
Hip Real

Toe Simulated
Step Simulated

Toe Real
Step Real

Position (m)

Position (m)

0.2

0.1

0

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

1
1.5
Time (s)

2

2.5

Velocity (m/s)

Velocity (m/s)

0.6
0.4
0.2
0

0.2
0

-0.2
0

1

1.5

2

2.5
500

Ankle
Hip

Toe
Step

Torque (Nm)

2
Force (kN)

0.5

1.5

1

Ankle
Hip

Toe
Step

0
-500

0

0.5

1
1.5
Time (s)
(a)

2

2.5

0

0.5

(b)

Figure 8. The fitting between robot data and the reverse-engineered controller. (a) The x position, velocity, and required force. (b) The
z position, velocity, and required torque.

96

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

JUNE 2020
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
