IEEE Robotics & Automation Magazine - June 2020 - 111

Experimental Results
To test our framework, we designed a similar pickup/dropoff scenario (see Figure 6) in which the quadrotor was tasked
to visit different points in the pickup area and return to the
drop-off location every round while avoiding two obstacles
(Figure 7). Real flights were performed with an AscTec
Hummingbird quadrotor controlled through the Robot
Operating System. A Vicon motion capture system was used
to track the position of the quadrotor and provide ground
truth position information. Two industrial fans blew wind in
the middle of the area, creating a disturbance toward the
obstacles. To generate safe and unsafe labels for the real scenario, 14 positions equidistant from one another in the pickup area were considered. For every avoidance distance
ra ! " 0.4, 0.5, 0.6, 0.7, 0.8 , m, trajectories were generated
from each starting position to the drop-off goal, and viceversa, and safety decisions about these routes were made
using a similar approach explained in the "Reachability Analysis" section. A trajectory was labeled unsafe if the maximum
deviation from the desired course became larger than the
distance between the obstacle and the desired route at any
point along the path. We trained two NNs, one for each subtask. In Figure 6, the safety decisions are shown when the
training set is provided as input to the NN.
During testing, we exploited the trained NN by executing
multiple passes back and forth between different points in
the pickup area through an unknown disturbance, generated in the middle of the area, that could push the quadrotor
toward the obstacles. Throughout the experiment, the vehicle was tasked to navigate using the smallest avoid distance ra = 0.4 m, if possible. As expected, the NN generated
unsafe outputs for some of the points. Consequently, ra was
increased by 0.1-m increments until a safe decision was
returned by the NN before sending the vehicle to the goal.
The computation time for the NN to make a safety decision
was on the order of milliseconds and constant for all trajectories, making it suitable for online replanning operations.
Conversely, the simulation-based reachability used during training is not suitable at runtime, as its computation
time increases linearly with the duration of the trajectory.

Navigation in Cluttered Environments
The other case study that we demonstrate in this article is a
navigation operation in a cluttered environment, such as a
heavily forested area. A UAV was tasked to reach a goal position in an area in which obstacles were scattered in a priori
unknown locations discoverable at runtime as the system
moved toward the final objective. To plan safety-guaranteed
trajectories, we considered a smaller environment with only
one obstacle in the center and trained and verified an NN to
predict the safety of the routes within this smaller region. The
trained and verified primitive space could then be fitted and
composed multiple times at runtime to assess safety in larger
regions with more obstacles.
Throughout a mission, every time the UAV encounters an
obstacle along its path, it selects an intermediate goal point
around the obstruction and queries the trained NN about
the safety of the trajectory to the selected target. If unsafe, a
new intermediate goal is queried until the output of the NN
is safe. This procedure is repeated multiple times for each
obstacle along the path until the vehicle reaches the final target position.
Offline Training
To train the NN for this operation, we used a small, box-shaped
workspace with one obstacle in the center, as shown in Figure 9. We picked 44 initial and 44 final positions uniformly
distributed across the start (to the left of the obstacle) and goal
(to the right of the obstacle) regions. The safety of the trajectories generated from those 1,936 start-goal position pairs

Actual Path
Desired Trajectory
Waypoints
Goal
Obstacles

2
1
y (m)

Figure 7 shows the sequence of snapshots, the decisions of
the NN, and the comparison between the desired and actual
trajectories (upper rows) for two rounds of the operation.
Finally, we ran an experiment in which we generated trajectories with the minimum avoid distance ra = 0.4 m, using
the wind disturbance from the previous experiment and disregarding the decision from the NN. During the first round,
which was predicted to be unsafe with ra = 0.4 m, the quadrotor collided with an obstacle (Figure 8), confirming the unsafe
decision predicted by the NN.

0
-1
-2
-4

-2

0
x (m)

Figure 8. The trajectory followed by the quadrotor for the first pickup task in Figure 7(a), with ra = 0.4 m and marked unsafe, resulting
in a crash.

JUNE 2020

IEEE ROBOTICS & AUTOMATION MAGAZINE

111

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020