IEEE Robotics & Automation Magazine - June 2023 - 89

IMU data is left to future work. Hardware connections are
shown in [2, Fig. 5].
This platform differs from existing tumbling robots in its
simple, blocky design, which allows for more robot parameters
to be easily modified in simulation. Randomized Unified
Robot Description Format (URDF) models
can be generated programmatically, increasing
the scope of possible domain randomization.
We generate a new robot each time one
is spawned in simulation during training by
randomizing 37 total parameters. Each generated
tumble bot has varying dimensions,
inertia characteristics, motor responses, and
friction coefficients. Every parameter is randomized,
with a standard deviation of 5% of
the nominal values. This amount of domain
randomization is necessary to ensure that
the trained policy is robust to small errors
in these parameters. Since the real tumble
bot is within the subset of these generated
robots, the policy is expected to transfer well
to the real world.
stick node. While IMU data were not used, the motion capture
node could be swapped for the IMU node. This would
shorten the time required to move the lab experiments to an
outdoor environment.
"
REINFORCEMENT LEARNING
ALLOWS FOR THE
DEVELOPMENT OF SOPHISTICATED
CONTROL
SCHEMES THAT CAN
ADAPT TO DIVERSE ENVIRONMENTS.
„
SOFTWARE
For
the physical prototype, the main onboard computer is a
Raspberry Pi 3 Model B+ running Ubuntu 16.04 with ROS
Kinetic. It is configured as an access point executing motor
commands produced by the trained network running on a
separate computer and receiving a stream of Vicon motion
capture data. The tumble bot uses an Arduino Uno as a motor
controller between the Pi and servos to allow for future
expansion and control of different servos. We chose the Raspberry
Pi for its compatibility with baselines models. For
future work, all computation will be performed by the Raspberry
Pi, with the only external communication being set
point commands.
The flow of data is shown in [2, Fig. 6]. We configured the
training environment using OpenAI Gym. For each episode
step, the servo outputs are sent from Gym, using a TCP socket.
The robot returns a position and orientation observation by
using the most recently received motion capture data. A timer
is used to ensure that communication is performed no faster
than the rate it was modeled, as in simulation. By choosing a
relatively slow rate of 1 Hz, effects of communication latency
are minimized. We designed the system for a latency of about
20 ms, allowing theoretical rates of about 50 Hz.
The system can easily switch between manual and autonomous
control, with a safety override to stop movement. This
was accomplished through the use of ROS. ROS is a communication
framework that allows different robots to use the
same software packages and makes it easy for these packages
to interact with one another. This package and node system
enables a modular software approach. The tumbling robot's
servo outputs are controlled via commands from the Gym
environment though the Gym link node when they are not
being manually controlled with a joystick through the joyPOLICY
TRAINING
Training a policy in simulation rather than
the real world decreases training time [9] as
long as the reality gap between the simulated
and real environments is not too large.
SIMULATION ENVIRONMENT
Simulation was performed using PyBullet, a
Python interface to the Bullet physics
engine. The simulation's numerical solver's
rate was left at the default of 240 Hz.
Between every action output, the simulation
was stepped 240 times. This results in an
action frequency of 1 Hz. While a much
higher frequency is possible, it is not
required for effective locomotion. Each
PPO episode was limited to 20 steps, which corresponds to
episodes of 20 s and 4,800 simulation steps.
The simulator was sensitive to changes in the solver's
rate due to the many collisions involved in the simulation
(a)
(b)
FIGURE 3. The two robots used in simulation and real-world
experiments. (a) The tumbling robot model in PyBullet simulation.
(b) The tumbling robot prototype.
JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
89
IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
