IEEE Robotics & Automation Magazine - December 2021 - 18

row index i and a controller with column index j, namely, to
D (, ).
KL ij
insights. For example, for a given staircase traversal direction
and total reward design, the two robots develop different policies.
This can be particularly seen in divergences (1,4) and
Task
rr A KL divergence that is low for two policies
implies that very similar actions are chosen for the same
observations.
The heat map facilitates the extraction of some very useful
(2,5) and demonstrates the importance of separate controller
training for the robots. Despite the fact that, in simulation, the
robots share the same motors, observations, and action space,
the differences in geometry and mass distribution result in
various necessary flipper actions throughout the traversal.
This further suggests that an operator sufficiently skilled to
control a given platform will not necessarily be able to use a
slightly different one.
We can additionally observe that for a given robot and
KL
Jag-Asc-3-Def 1
Jag-Asc-3-COG 2
Jag-Des-3-Ang 3
Abs-Asc-3-Def 4
Abs-Asc-3-Cog 5
Abs-Des-3-Ang 6
Divergence
0.6
0.5
0.4
0.3
0.2
0.1
Figure 7. The mean KL divergence between policies.
traversal direction, alternating the total reward function
induces a quantitative change in the employed actions, for
example, in divergences (1,2) and (4,5). This justifies our
interest in optimizing staircase negotiation with more elaborate
criteria than merely arriving at the goal. Finally, the
highest divergences are noticed between policies where the
staircase traversal is different, namely, for (1,6) and (2,6).
This clearly suggests that staircase ascent and descent require
independent treatment, as dynamics and risks significantly
differ. Two pairs of policies were unexpectedly found to be
similar, however, i.e., (1,5) and (3,4), an event that can be
attributed to the training stochasticity.
We performed a quantitative analysis of policy adequacy
on different platforms; see Figure 8. This experiment was
designed to assess how well a policy trained on one robot
would perform when deployed on another. We performed 10
such trials (rollouts) for a given task. Policies Jag-Asc-3-Def
80
60
40
Jag-Asc-3-Def
Task ID
0.22
0.21
0.2
0.19
0.18
Jag-Asc-3-Cog
Task ID
0.3
0.25
0.2
Jag-Des-3-Ang
Task ID
(a)
Abs-Des-3-Ang
0.3
0.25
0.2
Jag-Des-3-Ang
Task ID
(b)
Figure 8. The performance evaluation of policies deployed on the (a) Absolem and (b) Jaguar robots.
18 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2021
Abs-Des-3-Ang
Abs-Asc-3-Cog
0.09
0.08
0.07
Jag-Asc-3-Cog
Task ID
Abs-Asc-3-Cog
Abs-Asc-3-Def
60
50
40
30
Jag-Asc-3-Def
Task ID
Abs-Asc-3-Def
Task
Angular Velocity (rad/s)
COG Deviation (m)
Time Steps
Jag-Asc-3-Def 1
Jag-Asc-3-COG 2
Jag-Des-3-Ang 3
Abs-Asc-3-Def 4
Abs-Asc-3-Cog 5
Abs-Des-3-Ang 6
Angular Velocity (rad/s)
COG Deviation (m)
Time Steps

IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021