IEEE - Aerospace and Electronic Systems - July 2022 - 38

A Vision-Based Guidance Algorithm for Entering Buildings Through Windows for Delivery Drones
variables of interest in this section. They include image
frame fig, vehicle body frame fbg, and the window portal
frame fpg. The guidance procedure of the MAV through
the window center is splitted into distinct sections.
Approaching the window is done using a steering strategy.
Whenever DAE positioning is available, verticalization
comes into the process by generating lateral commands.
Figure 8.
(a) Drift in tracked area. (b) Tracked area after autocorrection.
STEERING GUIDANCE
Implementing optimal t per frame to detect exact
portal boundaries colored red in Figure 8, it is possible
to deploy the perspective-n-point algorithm [31] to
estimate accurate metric position data w.r.t a coordinate
system fixed in bottom-left corner of the entrance
frame (reprojected and visualized in Figure 8). Of
course, a prerequisite would be the metric dimensions
of the window frame and also accurate calibration
data [32] of the camera.
Another advantage ofDAE is the capability ofcorrecting
the tracker drift during the mission. This capability is
referred to as the Tracker Auto-Correction in Figure 2. As
illustrated in Figure 8(a), the tracked box has drifted from
the window portal during the long-time tracking process.
Based on the distance between the tracked box (blue color)
centroid and the window portal (red color) centroid, the
drift is measured framewise. If the drift remains more than
a certain value for a certain number of sequential frames,
the tracker is reinitialized on the smallest box surrounding
the window portal. Figure 8(b) shows the tracked box just
after this autocorrection. This autocorrection enhances the
tracking process by reducing the Median Flow tracker
drift, thus improving its utility for the specific task of window
entrance. Figure 9 shows how the DAE and Median
Flow complement each other; Median Flow provides an
initial search zone (a rect) for DAE to find the exact portal,
and DAE provides an exact reinitialization in the case of
tracker drifts. Also, the DAE parameter is recalculated
each 10 seconds, since as the robot gets closer to the building
the lighting condition might change.
GUIDANCE
The different coordinate systems used in this work are
drawn in Figure 1 and appeared in the superscript of the
in which
ei
y ¼ yi
t yi
c:
(13)
The MAV heading is controlled to face the window center
continuously. A PI controller is implemented and the
heading command _cb
c_ b
in which ei
cmd ¼ KP;cei
zontal center xi
image frame
cmd is derived as
Z
x þKI;c
ei
x
(8)
x is the pixel distance between the image horic
and the window portal center xi
ei
x ¼ xi
t xi
c:
t in the
(9)
Throughout this section, all controller gains of the form
Kx;x are some positive scalars.
For forward movement, whenever the MAV is aligned
to the window, a constant velocity (Vforward) command
is applied
Vb
x;cmd ¼ KP;xVforward
where
KP;x ¼
1;
if ei
x ethreshold
0; otherwise.
vertical reference yi
(11)
points exactly toward the front of the MAV. The _zb
calculated to align the vehicle vertically
z_b
cmd ¼ KP;zei
y þKI;z
Z
ei
y
In the case of installation direction for the camera, the
c should be determined such that it
cmd is
(12)
(10)
Figure 9.
Flowchart for combination of the Median Flow tracker with the DAE method.
38
IEEE A&E SYSTEMS MAGAZINE
JULY 2022

IEEE - Aerospace and Electronic Systems - July 2022

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2022