IEEE Robotics & Automation Magazine - June 2020 - 109

ra = 0.4 m
1
y (m)

y (m)

1
0

0
-1

-1
-2

0
x (m)

2

-2

(a)
ra = 0.5 m
y (m)

y (m)

0

2

0
x (m)

2

0
-1

-1
-2

0
x (m)

2

-2

(g)

(b)

ra = 0.6 m

ra = 0.6 m
1
y (m)

1
y (m)

0
x (m)
(f)
ra = 0.5 m

1

1

0

0
-1

-1
-2

0
x (m)
(c)
ra = 0.7 m

2

-2

0
x (m)
(h)
ra = 0.7 m

2

-2

0
x (m)

2

1
y (m)

1
y (m)

0

0
-1

-1
-2

0
x (m)
(d)
ra = 0.8 m

2

(i)

ra = 0.8 m

1

1
y (m)

Verification Results
Verification was performed with
the methods in the "Verification" section. Here, we present the results for
the second drop-off task shown
i n F i g u re 4, namely, the case in
which the UAV starts in the set
x 0 ! [0, 1], y 0 ! [- 0.5, 0.5] and aims
to reach the goal at [4, 0] m, with
ra = 0.45 m and disturbances d x,
d y ! " - 0.1, 0.1 , m/s.
The verification results are presented in Figure 5. We divided the initial
set into smaller subsets and verified
each one separately to keep the approximation error in Flow* small enough.
The size of these subsets [i.e., the 5-cm
boxes in Figure 5(b)] was chosen after
some preliminary testing; these subsets
were large enough to verify the majority of the initial set with a small approximation error. Some subsets were
further refined when an instance
resulted in an error that was too large.
Refinements were necessary at the
NN's decision boundary as well as for
sets that triggered multiple if-cases in
the planner (e.g., when planning
around multiple obstacles). The verification was performed using Amazon

ra = 0.4 m

y (m)

distance marked on top of the figure.
Due to space constraints, we show
examples of data for only three avoidance distance values: ra = 0.3 m,
ra = 0.45 m, and ra = 0.7 m. Arrows
inside the workspace indicate the
direction of motion of the vehicle.
Inside each subfigure, the green (red)
dots represent initial positions from
which the trajectories to the goal are
labeled safe (unsafe) using reachability analysis. The green (red) circles
around the dots denote the decisions of the NN on the same training
points. As can be noticed and as
expected, when the avoidance distance increases, the number of safe
initial positions also increases in
both missions because the distance
between the desired trajectories and
the obstacles becomes larger. Therefore, increasing the avoidance distance improves safety; however, the
routes become longer, which generally
is not desirable due to energy concerns.

0
-1

0
-1

-2

0
x (m)
(e)

2

-2

Safe Labels
Unsafe Labels
NN Safe Labels
NN Unsafe Labels
Obstacles
Drop-Off Position

0
x (m)
(j)

2

Figure 6. The safe and unsafe training and NN results for the pickup and drop-off tasks
in the area where experiments were performed. An FP (red dot inside a green circle)
corresponds to an unsafe label (red dot) with a safe NN decision (green circle), while
an FN (green dot inside a red circle) is a safe label (green dot) marked as unsafe (red
circle). The number of FPs is zero for both NNs, and there are two and zero FNs for the
first and second NN, respectively. The drop-off tasks where (a) ra = 0.4 m, (b) ra = 0.5
m, (c) ra = 0.6 m, (d) ra = 0.7 m, and (e) ra = 0.8 m. The pickup tasks where (f) ra =
0.4 m, (g) ra = 0.5 m, (h) ra = 0.6 m, (i) ra = 0.7 m, and (j) ra = 0.8 m.

JUNE 2020

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

109
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
