IEEE Robotics & Automation Magazine - December 2021 - 19

(a)
(b)
Figure 9. Staircase negotiation with congruent control of 5 DoF. (a) Ascent by minimizing the COG deviation. (b) Descent by
minimizing the pitch angular velocity.
and Abs-Asc-3-Def are evaluated on the basis of time steps in
a rollout. As we can see, if we deploy a policy on a platform
for which it was trained, the robot spends fewer episodes until
task completion. Characteristically, rollouts tested on Absolem
last around 47 time steps if we use Abs-Asc-3-Def,
against 80 time steps if Jag-Asc-3-Def is employed, where performance
is significantly degraded. Similarly, deploying JagAsc-3-Def
on Jaguar requires, on average, 39 time steps as
opposed to 51 when Abs-Asc-3-Def is applied.
Deploying Jag-Asc-3-Cog on Absolem further reveals a
slight performance degradation in terms of COG deviation,
measured at 0.206 against 0.188 when Abs-Asc-3-Cog is
deployed. In contrast, the deployment of these two policies on
Jaguar seems equally performant. With respect to descent, the
deployment of Jag-Des-3-Ang and Abs-Des-3-Ang on the Jaguar
robot clearly favors the robot-specific policy, but the same
policies perform equally well on Absolem. We can conclude that
a policy trained and deployed on the same robot provides the
best performance; otherwise, the results can only be degraded.
Real-World Performance
Policies Jag-Asc-5-COG and Jag-Des-5-Ang were tested on
two staircases, as presented in Figure 9 and Table 3 (we recall
that the arm is actively involved in the optimized controllers).
The respective experiments are included in the supplementary
video (available at https://doi.org/10.1109/MRA.2021
.3114105). Overall, we performed 10 trials for each task ID
(five trials per staircase) discussed in this section, all of which
were successful.
Jag-Asc-5-COG was tested on a big staircase, with the
robot completing the task by keeping its rear flippers down
and its front flippers up; the first arm joint was inclined clockwise,
and the second was counterclockwise. The flipper configuration
ensures that the robot has contact points with rear and
front stair steps. The entire arm is moved forward to decrease
COG deviation, which minimizes tip-over risks and the
chance of getting stuck. A point of ambiguity emerges for the
arm, as certain configurations decrease Cy
and increase Cx
at
the same time and vice versa; hence, minimizing COG deviation
D is not straightforward. Remarkably, the obtained results
of the learned controller show that the robot learned to incline
the first arm link forward and the second link backward,
which is, indeed, the best configuration. For reference, Table 4
provides average observed values for some key parameters.
The accomplishment of descent task Jag-Des-5-Ang on the
big staircase merits more attention. One of the most important
aspects is the linear velocity control (see the supplementary
video), where the robot moves smoothly and adapts its speed
very attentively, which leads to increased time spent completing
the staircase traversal. Naturally, this greatly reduces the mean
perceived angular velocity by 0.1 rad/s, as opposed to the ascent
task. The ascent and descent tasks were further successfully
accomplished on a small staircase composed of three steps,
each with a height 0.17 m and a length 0.305 m. The values
reported in Table 4 enable us to draw the same conclusions for
ascent and descent that we did for the big staircase.
Conclusion
We presented an effective RL framework for control learning
of staircase negotiation in multiple task variants. In particular,
we trained controllers for two robotic platforms through simulation,
with reward function designs suited for ascent and
descent and different staircases. Our results show learning
convergence and optimization of targeted behavior features
for both platforms within 150 episodes. We employed KL
Table 3. The staircase configurations.
Name
Big
Small
Number of Steps Height (m)
Five
Three
0.195
0.17
Length (m)
0.275
0.305
Table 4. The observed mean target values on
real staircases.
Task ID
Stair Cy (m)
Jag-Asc-5-COG Big
D (m)
Angular
Velocity
(rad/s)
0.11 ± 0.01 0.12 ± 0.01 0.33 ± 0.06
Jag-Asc-5-COG Small 0.1 ± 0.01 0.11 ± 0.01 0.36 ± 0.09
Jag-Des-5-Ang Big
0.16 ± 0.02 0.09 ± 0.02 0.23 ± 0.06
Jag-Des-5-Ang Small 0.12 ± 0.02 0.05 ± 0.02 0.23 ± 0.09
DECEMBER 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
19

https://doi.org/10.1109/MRA.2021.3114105 https://doi.org/10.1109/MRA.2021.3114105

IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021