IEEE Robotics & Automation Magazine - December 2022 - 100

Second, the power consumption results of the learned
gait are marked with red points in Figure 3. The data depict
a linear relationship between travel velocity and power
consumption, which is in line with the physical law
.
PowerForce Speed#
=
There are only a few points with
higher power consumption when the velocity is below 0.08
m/s. Importantly, this also reveals the adaptability of the
learning approach for generating gaits in a range of velocities.
It can also be observed that the mean velocities do not exactly
align with the specified interval of 0.005, especially at higher
target velocities. The reason for this is the difficulty to achieve
the exact ratio between holding the right velocity and performing
the corresponding power-efficient locomotion.
Comparison
We first have a look at the energy metric based on averaged
power per velocity. After putting the power efficiency data of
the gait equation controller (see Figure 3) and the NN-based
controller together, we can clearly conclude that the NN-based
controller has a much better power efficiency at a range
of velocities. As the velocity grows, the advantage of the
NN-based controller for saving energy is even more obvious.
To show that the proposed method can generate energyefficient
gaits in a more general context, we also evaluated
both controllers in different environments (friction coefficient)
and with different robot designs (the number of body
modules) in simulation. The results are visualized in Figure 5.
In Figure 5(a), we change the robot's design by adding or
reducing body modules to test the proposed method. For a
snake robot with a joint number range from six to nine (the
default number is eight), the proposed method shows consistent
performance superiority to generate many energy-efficient
gaits across different traveling speeds. In Figure 5(b), we
change the friction coefficient of the environment to test
effectiveness of the proposed method. On one hand, results
clearly show that the NN-based controller can generate gaits
that consume less energy than the gaits generated from an
equation-based controller. On the other, the NN-based controller
also shows a much more stable performance in generating
energy-efficient gaits, while the equation-based method
exhibits fluctuating energy-consumption results.
The NN-based controller improves upon the traditional
kinematic-based one in two ways. On one hand, the traditional
gait equation controller is based on kinematics and describes
the gait movement with no influence of physical forces such as
friction or damping. It only extracts several critical parameters
to represent the gait, while the interaction between the snakelike
robot and the environment is highly complex. Although
the parameterized gait equation simplifies the control task, it
inevitably brings difficulties for designing more sophisticated
gaits despite using traditional optimization technologies (the
grid-search algorithm), especially when the parameter space
will grow exponentially with the increasing joint numbers.
Conversely, the RL method shows its effectiveness in its ability
to solve this kind of complex control problem as it is trained
directly in the dynamic environment. It is able to generate
undulation gaits that not only imitate the movement of real
snakes but also explore its limitations while continuing to
improve its behavior under its hardware constraints.
Prototype Experiments
Sim-to-Real Transfer Framework
To make it feasible to transfer a learned policy from simulation
to the real world, we should design a framework to easily
connect the NN-based controller to the simulation environment
or the real world with the same configuration. As
shown in Figure 6, the NN-based controller has an interface
1,400
1,200
1,000
800
600
400
1,400
1,200
1,000
800
600
400
0.05
0.1
0.15
0.2
6
(a)
Equation-Based Controller
7
8
9
0.05
0.1
0.15
0.2
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2
(b)
NN-Based Controller
Figure 5. The simulation results from generalized scenarios. (a) The cost-of-transport (COT) performance of gaits generated from
different robot design (body modules). (b) The COT performance of gaits generated from environments with different friction
coefficients. The solid line shows the gaits generated from the NN-based controller. The dashed line shows the gaits generated from
the gait equation controller. The error bars show the standard deviation of the energy consumption at each velocity.
100 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2022
COT (W/m)
COT (W/m)
Velocity (m/s)
Velocity (m/s)
Joint Number
Friction Coefficient

IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022