IEEE Robotics & Automation Magazine - December 2022 - 155

fine-tune the parameters of the recovery
planner but found it difficult to find a
single set of parameters to recover the
robot from all out-of-distribution scenarios
while not accidentally driving
the robot into such scenarios throughout
the entire course. On one hand,
the simple nature of the recovery planner
designed on site during the competition
contributed to the failure. On
the other hand, tuning the parameters
of a planner to cover as many scenarios
as possible remains a difficult
problem and will be discussed further
in the following.
Tunability of Classical Systems
Similar to Temple's rule-based recovery
planner, the UT Austin team's entire
navigation system relies on classical
methods: EnML localization, a mediumhorizon
kinematic planner, and a local
rollout-based kinodynamic planner.
Inevitably, these classical approaches
have numerous tuning parameters,
which need to be correctly tuned to
cover as many scenarios as possible. A
natural disadvantage of relying on a
single set of parameters to cover all
different difficult scenarios in the
BARN Challenge (e.g., dense obstacle
fields, narrow curving hallways, and
relatively open spaces) is the inevitable
tradeoff, or compromise, to sacrifice
performance in some scenarios to succeed
in others and to decrease speed
for better safety.
Indeed, the UT Austin team's strategy
in the physical finals was to spend
the first 20 min in the 30-min timed
session to fine-tune the system parameters
until a good set of parameters that
allowed successful navigation through
the entire obstacle course was found,
then finish three successful " safety trials "
first and, finally, retune the system
to enable faster and more aggressive
but riskier navigation behaviors to
reduce the average traversal time.
Although most such " speed trials "
failed, luckily for the UT Austin team,
other teams' inability to safely finish
three collision-free trials to the goal
made it the winner of the BARN Challenge
but only by having a higher success
rate (not faster navigation).
Two orthogonal future research
directions can potentially help with the
tunability of navigation systems: 1)
developing planners free of tunable
parameters on site during deployment,
such as end-to-end learning
approaches, but, as mentioned previously,
with significantly better
simulation-to-real transfer and generalizability
and 2) enabling more intelligent
parameter tuning of classical
systems, rather than laborious manual
tuning, for example, through automatic
tuning and even dynamic parameter
policies learned from teleoperated
demonstration, corrective interventions,
evaluative feedback, and reinforcement
learning.
Getting " Unstuck "
Although most of the failed trials during
the physical finals were due to collisions
with obstacles, there were also
many trials that did not succeed
because the robot got stuck in some
densely populated obstacle areas. In
those places, the robot kept repeating
the same behaviors multiple times,
e.g., detecting imminent collisions
with obstacles, rotating in place, backing
up, resuming navigation, detecting
the same imminent collision again,
and so on. Such behavior sometimes
led to a collision with an obstacle and
sometimes got the robot stuck forever,
although it might also have succeeded
on rare occasions. All three teams
experienced such behaviors, with the
UT Austin and UVA teams being able
to fix them by tuning parameters and
the Temple team changing the threshold
between DRL-VO and the recovery
planner.
Similarly, in real-world autonomous
robot navigation, how to get " unstuck "
safely remains a common and challenging
problem. No matter how intelligent
an autonomous mobile robot is,
it may still make mistakes in the real
world, e.g., when facing scenarios out
of the training distribution, corner
cases not considered by the system
developer, and situations where the
current parameter set is not appropriate.
It is very likely that the robot will
repeat the same mistake over and over,
e.g., getting stuck at the same place,
which needs to be avoided. Future
research should investigate ways to
identify such " stuck " situations, balance
the tradeoff between exploitation
and exploration (i.e., when to keep
trying the previous way versus when
to try out new ways to get unstuck),
utilize previous successful exploratory
experiences in future similar scenarios
to not get stuck again, and leverage
offline computation to correct such
failure cases in the future.
Tradeoff Between Safety
and Speed
While the BARN Challenge was originally
designed to test existing navigation
systems' speed of maneuvering
through highly constrained obstacle
environments, given the safety constraint
of being collision-free, it ended
up being a competition about safety
alone. The UT Austin team won the
competition simply by safely navigating
eight out of nine physical trials, not by
doing so with the fastest speed. All the
teams except the UT Austin team, after
it figured out an effective set of parameters
for each physical obstacle course,
struggled with simply reaching the goal
without any collision. The challenge
organizers also deployed the widely
used DWA planner in the ROS move_
base navigation stack in the physical
obstacle courses, only to find out that,
despite being relatively safe compared to
the participating teams' methods, it
struggled with many narrow spaces and
got stuck in those places very often.
Such a fact shows that the current
autonomous mobile robot navigation
capability still lags further behind than
one may expect.
Latency Compensation
for High Speed
Only the UT Austin team attempted
to pursue higher-speed navigation
(>0.5 m/s), doing so after an appropriate
parameter set was found for the
particular physical course and three
successful " safety trials " had been
achieved. However, most " speed trials "
ended in collision. One contributing
factor to such failure was improper
DECEMBER 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
155
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
