IEEE Robotics & Automation Magazine - December 2022 - 150

outperformed the baseline dynamic
window approach (DWA), with both a
2- and 0.5-m/s maximum speed, the
latter of which is the default local
planner for the Jackal robot. However,
only one approach (from Temple University)
outperformed all baselines.
The top three teams from the simulation
qualifier, i.e., Temple Robotics
and Artificial Intelligence Lab
(TRAIL), from Temple University;
Autonomous Mobile Robotics Laboratory
(AMRL), from the University of
Texas at Austin (UT Austin); and
Autonomous Mobile Robots (AMR)
Lab, from the University of Virginia
(UVA), were invited to the physical
finals at ICRA 2022.
Physical Finals
The physical finals took place at ICRA
2022, in the Philadelphia Convention
Center, on 25 and 26 May. Two physical
Jackal robots with the same sensors and
actuators were provided by the competition
sponsor, Clearpath Robotics.
Rules
Physical obstacle courses were set up
using approximately 200 cardboard
boxes in the convention center (Figure 2).
Because the goal of the challenge was to
test a navigation system's ability to perform
local planning, all three physical
obstacle courses had an obvious passage
that connected the start and goal locations
(i.e., the robot should not have
been be confused by global planing at
all), but the overall obstacle clearance
when traversing this passage was de -
signed to be very constrained, e.g., a few
centimeters around the robot.
While it was the organizers' original
Figure 2. One of the three physical obstacle courses during the finals.
Table 1. The simulation results.
Rank Team/Method (University)
1
2
3
4
5
6
7
8
9
10
11
Temple Robotics and Artificial Intelligence Lab
(Temple University)
LfLH (baseline)
Autonomous Mobile Robotics Laboratory (University of
Texas at Austin)
Autonomous Mobile Robots (University of Virginia)
E band (baseline)
End to end (baseline)
APPLR-DWA (baseline)
Yiyuiii (Nanjing University)
NavBot (Indian Institute of Science)
Fast (2.0 m/s) DWA (baseline)
Default (0.5 m/s) DWA (baseline)
Score
0.2415
0.2334
0.231
0.22
0.2053
0.2042
0.1979
0.1969
0.1733
0.1709
0.1627
DWA: dynamic window approach; LfLH: Learning from Learned Hallucination; APPLR: Adaptive
Planner Parameter Learning.
intention to run exactly the same navigation
systems submitted by the three
top teams and use the same scoring
metric in the simulation qualifiers in
the physical finals, these systems suffered
from (surprisingly) poor navigation
performance in the real world (not
even being able to finish one single trial
without any collisions). Therefore, the
organizers decided to change the rules
by giving each team 30 min before competing
in each of the three physical
obstacle courses to fine-tune their navigation
systems. After all three teams
had this chance to set up for a particular
obstacle course, the actual physical
finals started as a 30-min timed session
for each team. In each 30-min session, a
team tested its navigation system in the
obstacle course and notified the organizers
when it was ready to time a competition
trial. Each team had the
opportunity to run five timed trials
(after notifying the organizers). The
fastest three out of the five timed trials
were counted, and the team that had the
most successful trials (reaching the goal
without any collisions) was the winner.
In the case of a tie, the team with the
fastest average traversal time would be
declared the winner.
Results
The physical finals took place on 25 and
26 May (see the final award ceremony
in Figure 3). The three teams' navigation
performance is provided in Table 2.
Since all navigation systems navigated at
roughly the same speed, the final results
were determined solely by the success
150 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2022
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
