IEEE Robotics & Automation Magazine - December 2022 - 99

network is eight. As the head velocity is one overall property
of the robotic system, the jointwise or nonjointwise style is
not applicable here. The inputs for predicting these three
states are the joint position, action, and target velocity.
To enable learning of different velocities, the parameter vt
is changed by iterating over 0.05, 0.1, 0.15, 0.2, and 0.25 for
each episode while training. It should be noted that in theory,
we can also select target velocity from a uniform distribution
to train the network. But empirical results show that the training
is much more successful and stable when using discretized
target velocities. Meanwhile, to simplify the
beginning of the learning process, the first 100 episodes are
trained with a fixed target velocity of 0.1 m/s.
Simulation Results and Comparisons
Baseline Performances
The power consumption and corresponding velocity results
from the grid-search algorithm are shown in Figure 3 as a
point cloud of parameter sets using dot markers. The lowest
points at different velocities in the point cloud have the highest
energy efficiency. To reduce the number of equation
parameters, we select five temporal frequencies to show the
pattern of gaits generated by the gait equation as temporal frequency
is the most direct factor that impacts velocity. Second,
instead of connecting the scatter points with lines, we show
the confidence region of the gaits generated with each frequency.
We can observe that low frequency (0.25 Hz) leads to
very slow gaits, while high frequency (2 Hz) leads to energyintensive
gaits. The gaits with 1.5 Hz are more promising for
generating energy-efficient gaits. We also demonstrate that
the grid-search method suffers from being inefficient to
search proper parameters for energy-efficient gaits as most of
the parameter sets in the searching space are distributed in
the low-velocity area (. ).001m/s
-
Note that we show all the
gaits from the grid-search algorithm as we want to depict the
inefficiency of the baseline method.
NN-Based Controller Performance
In this study, target velocities in the range of [. ,. ]
0025 025m/s
with a step interval of 0.005 m/s are used for the evaluation.
The simulation results demonstrate that the NN-based controller
has succeeded in learning a series of gaits from scratch
without knowing any prior locomotion skills.
First, the NN-based controller can perform very accurate
locomotion gaits in terms of velocity even though the NNbased
controller is trained with only five target velocities (0.05,
0.1, 0.15, 0.2, and 0.25 m/s). As shown in Figure 4, the targeted
velocities are represented by blue solid dots, and the measured
velocities from the simulation are marked by red solid dots.
We can observe that the measured velocities almost match the
target velocities from 0.05 to 0.20 m/s with only very small
errors close to zero. In the low-velocity range (less than
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
