IEEE Robotics & Automation Magazine - June 2023 - 87

types of robots can be effective in applications where a highmobility-to-size
ratio is required [1]. It is often the case that
real-world environments include complex terrains composed
of varying surfaces, rapid changes in elevation, and impassable
obstacles. Tumbling locomotion can allow robots to traverse
these complex environments in which traditional
locomotion fails. However, this comes with the cost of being
difficult to control. For example, interactions with the terrain
can be used to efficiently maneuver down a hill, but they can
also cause the robot to roll in unanticipated directions. All
these characteristics make tumbling robots an attractive
option for mobile robot applications, and they open an interesting
area of control research where a robot's motion cannot
be modeled or predicted.
Tumbling motion is defined as being stochastic: for a given
motor command, we are not able to predict the motion of the
robot until after it has executed a tumble. Even seemingly
simple maneuvers, such as driving forward, require constant
adjustment, as variations in surface friction and terrain height
can easily perturb the robot significantly from its desired path.
As a result, tumbling robots often need to segment their movement
into discrete tumbles [2], where they can analyze the
result of their previous action and correct for any disturbances
that caused them to deviate from their goal. This represents
a challenging control problem, as there is no direct mapping
from the disturbance to a motor command that can correct for
it. Tumbling robots require a much more sophisticated control
policy that has become feasible only in recent years, due to
advancements in RL.
Our previous work in [2] takes the first steps of exploring
the importance of RL for tumbling locomotion (see Figure 1).
It proposes the use of RL methods for tumbling robot control,
due to controller complexity and environmental uncertainty.
In this article, we examine how well RL policies perform in
comparison with a simple controller and a random policy.
Also, we test the RL policies in different environments than
they were trained in and measure their performance. We focus
on the task of set point navigation and evaluate trained policies
on three different environments: flat ground, an uneven
surface, and valley-hill terrain (shown in Figure 2). The testing
of the policies takes place in simulated and real-world
environments, demonstrating promising results for deploying
tumbling robots outside of the lab.
RELATED WORK
CONTROL OF TUMBLING ROBOTS
The benefits of tumbling locomotion include terrain-body
interactions that allow tumbling robots to successfully navigate
many terrains with few actuated degrees of freedom
(DoF) [3]. As discussed in [2] and [3], tumbling locomotion
has several advantages over other methods, such as simple
hardware and increased mobility relative to size. These
characteristics have led to tumbling robots being designed
for applications that require small and light robots, such as
underwater sampling [4] and jumping over obstacles [5].
More importantly for our application, tumbling locomotion
has proved to be a difficult control problem that we can use
to test the capabilities of RL-based control policies.
(a)
(b)
FIGURE 1. One of our tumbling robot prototypes navigating to a
set point (a collage of several frames from a physical trial).
FIGURE 2. The two different terrains used for the real-world
experiments. (a) The uneven surface constructed with randomly
placed foam tiles. Each tile is 1 ft × 1 ft × 0.25 in. (b) The valley-hill
terrain constructed with foam tiles. The maximum terrain height is
10 in. Each tile is 0.5 ft × 0.5 ft × 0.5 in. A tarp (not pictured) was
laid over the terrain during physical trials to prevent the tumble
bot's legs from getting stuck in gaps between the tiles.
JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
87
IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
