IEEE Robotics & Automation Magazine - December 2022 - 98

the given joint position .jz The input of the network is the
joint position .iz The informative network is trained using
supervised learning, where the dataset is directly sampled
from simulation.
Data Generation
With a well-trained NN-based controller, the agent is controlled
to slither in the simulation environment with a varying
target velocity to sample training data. Each episode contains
400 time steps, the target velocity starts from 0.05 m/s and
slightly increases . e133m/s
-3
at each episode until 0.25 m/s.
One logged data point contains the target velocity, joint position
of the timestamp, and values of the last action of the NNbased
controller. The generated data are shuffled and split,
where 80% of the dataset is used for training and the other
20% is used for testing.
Informative Network Model
Three observation features, namely, the joint velocity ,
.
zj
joint torque ,jx and head velocity ,v1
used for training the
NN-based controller in the simulation are not available in
the real world. Because each feature has its own characteristic,
we design three separate prediction models to predict
them with the same architecture. For one prediction network,
the input consists of the action values and the joint
positions for each joint in the last two time steps, together
with the target velocity.
As explained in [24], dynamics of the actuators are independent
of each other and, accordingly, they trained their
models for predicting actuator forces separately. However, for
the snake-like robot, we believe that the action of one joint
may affect other joints because the snake robot is a class of
Training
serially connected, active-cord mechanisms. Therefore, we
adopted two approaches for sampling the input data: the
jointwise and nonjointwise models. For a jointwise model,
each value of the feature is strongly related to one's specific
joint index, e.g., the torque of the first joint is predicted using
only the data from the first joint. The jointwise prediction
models split the input features of the input layer by joint and
process them separately. The processed values are concatenated
at the output layer. Oppositely, the nonjointwise models
process values from all the existing joints together at once as
input and predict output features for all the joints at once. The
main assumption of this is that a target feature depends on the
entire kinematics of the snake-like robot.
In this work, prediction models with two layers of multilayer
perceptron long short-term memory (LSTM), or a single
Temporal Convolutional Network layer (which consists of
multiple convolution layers) were tested. For the length of
input sequences (the number of past time steps), two, four,
and eight were used for training to observe how the inputsequence
length affects model performance. The meansquare
error was used as the loss function for training models,
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
