IEEE - Aerospace and Electronic Systems - July 2022 - 35

Fahimi et al.
tracked box area is used as the proximity measure. As this
measure increases up to a certain value, the DAE algorithm is
applied. From now on, exact portal section boundaries
extracted by the DAE method is used to correct the tracker in
the case of drifting from the portal section. The details about
the functionality of DAE to extract exact boundaries and
tracker corrections are discussed in the section " Dark Area
Extraction " section. " Furthermore, the exact portal boundaries
are also used to obtain more accurate navigation data, i.e., 6DoF
positioning data, in which the position and orientation of
MAV in the portal coordinate system is determined, as
depicted in Figure 1. The deviation ofthe vehicle position vector
expressed in the portal coordinate system from the window
surface normal direction (cp
cv angle) is then corrected by
applying lateral velocity commands (Vy command). Finally,
when the MAV loses sight of boundaries (practically in less
than a 1-meter distance to the window), it is guided to an estimated
straight path through the center ofthe portal section.
DARK AREA EXTRACTION
We start by motivating the idea of dynamic thresholding
followed by the introduction of a thresholding parameter
to detect portal section boundaries. We then propose an
optimization method to obtain some candidate values for
the parameter leading to a reliable estimation of the
boundaries. Then, the most optimum value is determined
through minimization of a cost function.
DYNAMIC THRESHOLDING
The objective of the DAE is to extract the boundaries of the
window portal in an image segment provided by the tracker,
with a few pixels offset around the tracked box. Figure 3(a)
shows a typical example of such segment. The method is
based on the observation that under daylight the least reflective
part of the image segment is most likely associated with
the window portal. As such, an effective thresholdingmethod
could extract this particular area, which in turn leads to an
estimation ofthe portal boundaries [see Figure 3(b)].
Unfortunately, it is revealed that a constant preset
threshold value will not always extract the portal section.
This is the case even between the consecutive frames of
the MAV's camera stream. As the MAV position changes
during the flight, the light condition might substantially
differ between frames. Also, the light reflection of the
building facade is subject to change in different distances.
Another source ofdiscrepancy is the appearance ofobjects
possibly located inside the window portal and become visible
in near distances. All these situations lead to defining
the threshold value as a frame-dependent parameter. It is
clear that the manual adjustment of this parameter in each
frame is impractical. As such, we adopt a dynamic thresholding
approach as
JULY 2022
SEARCHING FOR LOCAL OPTIMUMS
We first present a method that proposes a set of candidates
(denoted by Q) for the DAE parameter t. We then address
how the most desirable value can be determined from the candidates
set. A desirable DAE parameter corresponds to a
threshold value leading to a binary image containing a solid
white polygon for the window portal section with nojunctions
with any other white blob. Notice that in the binarization we
assign white color to the darker (less reflective) pixels.
An input intensity image (the value channel in HSV
space) is cropped around tracked box and then thresholded by
t to form a binary image BðtÞ. Figure 4 illustrates some samples
ofBðtÞ corresponding to a set of different t values. The
window portal is well distinguished using a certain range oft
values. To determine a t value as such, we define
sðtÞ¼
X
i
X
j
Bði; j; tÞ
(2)
as the sum of pixel values inside B, in which Bði; j; tÞ
denotes the pixel value in the ith row and the jth column.
We assume that the brightness between pixels inside the
window area does not differ as much as other parts of the
image; i.e., the brightness in the window area is more uniform.
IEEE A&E SYSTEMS MAGAZINE
35
threshValue ¼ð1 tÞminðVÞþ tmaxðVÞ
(1)
in which V denotes the value channel of the tracked image
segment in HSV space, and t 2½01 is defined as the DAE
parameter.
Our experiments show that t depends mostly on the
relative brightness between the building facade and the
window, while being fairly invariant to the light condition.
As such, it can safely be set for each mission only once at
the beginning. One possibility is to manually adjust the
value of t on the initial frame so as to achieve the extraction
of the window portal similar to Figure 3(b). However,
toward an autonomous entrance mission, we propose two
methods for automatically tuning this parameter, as presented
in the following sections.
Figure 3.
(a) Image segment containing the window portal. (b) Thresholded
image so that the window portal is extracted as a distinct region.
IEEE - Aerospace and Electronic Systems - July 2022

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2022
