IEEE Robotics & Automation Magazine - December 2022 - 96

cost of transport, which is the power P divided by the mass m,
the gravity g and the velocity v:
COT== .
mgd
E
mgv
P
(1)
This unit-less measurement is able to compare the efficiency
between different mobile systems. As the same robot and
environmental properties are compared, those constants only
scale the results and would not provide any additional insight
for the comparison. Thus, a simplified metric calculates the
averaged power per velocity (APPV) as
APPV Pv/ .=
Baseline Controller
In this section, we first introduce the widely used method for
generating gaits for snake-like robots, which is the gait equation
controller. Then we use the grid-search algorithm to
explore energy-efficient gaits at different velocities by brutally
searching the parameter space with fine steps.
Gait Equation Controller
As the most widely adopted method, the gait equation controls
the locomotion gaits of a snake-like robot by shaping its
back curve as a moving sinusoid wave that propagates along
the body. This method or similar ideas (e.g., CPG-based
methods) have been used for decades by many kinds of
snake-like robots to generate different gaits [2]. We utilize the
gait equation from [9], which is modeled as
z~$ sin
(,)( ).
nt
=+ +
`
n xy At n
$
N
j
m
(2)
z(, )nt presents the joint-angle value at time t, where n is the
joint index and N is the total joint number. m and ~ are the
spatial and temporal frequency of the movement, respectively.
The spatial frequency represents the cycle number of the
wave, and the temporal frequency represents the traveling
speed of the wave. A is the serpentine amplitude and x and y
are the constants for shaping the body curve.
Grid-Search Optimization
We take the grid-search algorithm to explore the parameter
space, which can lead to optimum gaits as long as the
Table 2. The gait parameters used for the grid
search.
Parameters Descriptions
~
Temporal
frequency
A
y
x
m
Amplitude (°)
Values
0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4,
1.6, 1,8, 2
30, 35, 40, 45, 50, 55, 60, 65,
70, 75, 80, 85
Linear reduction 0.1, 0.2, 0.3, 0.4
Linear reduction (1−y)
Spatial
frequency (°)
searching intervals are small enough. We use the grid-search
method to generate a variety of gaits and determine those
parameter combinations with the best power efficiency at different
velocities. The grid-search method generates a Cartesian
product from the parameters in Table 2, resulting in
10,080 parameter sets. Then, each motion parameter set gets
tested by running 1,000 steps in the simulation environment.
For each run, the first 200 time steps are ignored and the
remaining 800 time steps are evaluated for collecting experimental
data. It is because it has been observed that the snake
robot needs approximately 200 time steps to accelerate and
then moves at a steady speed.
NN-Based Controller
Observation Space
The full observation space consists of all the joint position ,jz
the joint velocity ,
velocity ,v1
.
zj
the actuator torque ,jx the head module
.
and the target velocity v .t The joint position jz
and its joint velocity zj are required to learn the locomotion
helps to sense its global velocity,
which offers better movement awareness. To learn an energyefficient
gait, the sense of energy consumption is necessary.
Therefore, the actuator torque of each joint jx is provided and
can be interpreted in combination with
.
total power usage. The specified target velocity vt
zj to determine the
is passed to
the environment and can be changed, which is required to
control velocity of the robot.
Action Space
The action space has 8 DoF with finite-continuous values in
the range of [. ,. ],15 15which
linearly translates to a corresponding
joint angle z in the range of [, ].
90 90
- cc Each
action represents eight actuator angle positions of the servo
motors. In the real-world setup, the joint can turn around
with the motor speed set as 0.07 s/60° and the gear ratio as
15 :.
143 The control frequency for the snake-like robot is set
as 20 Hz. Thus, the maximum turning angle in 0.05 s is set as
12° in the simulation. The simulation time step is also set as
0.05 s. It should be noted that 20 Hz is selected to ensure realtime
calculation of the NN-based controller.
Reward Function
The objective of our task is to learn an energy-efficient gait for
a variety of specified velocities. Therefore, the energy consumption
and the difference between the actual model velocity
and the target velocity are the main criteria to find a
successful behavior. The challenge is to combine the variables
into one numerical reward and weigh them for each time
step. Therefore, we first split the power efficiency and velocity
criteria into two normalized reward function components.
First, a normalized reward is defined to maintain the spec40,
45, 50, 55, 60, 65, 70, 75,
80, 85, 90, 95, 100, 105, 110,
115, 120, 125, 130, 135, 140
96 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2022
ified velocity. The objective is to reach and maintain the target
velocity vt
by comparing it with the head velocity v .1 The following
function represents the velocity reward:
and represent the proprioceptive awareness of the robot. The
head module velocity v1
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
