IEEE Robotics & Automation Magazine - September 2012 - 49

fusion component fuses the proprioceptive and exteroceptive sensor data. Mission-dependent recognition of world
features, such as persons or interesting objects, is done in
the recognition module. World representation, as well as
planning and decision-making functionalities, is realized in
the cognition layer. Lastly, the action layer is involved in stabilizing and moving the UAV in the desired manner. Such
categorization allows for a clear definition of the interfaces
between components and the minimum required set of
component functionalities. The realization of this structure
in our experimental system can be presented more clearly
when grouped by the layer subdivision.
Perception
The UAV should be able to fly in structured indoor environments as well as outdoors. The indoor environments consist
of clearly defined vertical structures (walls) that can be
detected by a laser scanner. However, poor lighting conditions
and a low number of environment features make indoor environments unsuitable for a camera-based odometry system.
Conversely, the outdoor environments lack clear structures.
Sunlit environments contain light in the part of the spectrum
that coincides with that used by infrared laser scanners, disturbing the measurements. This makes low-powered lightweight laser scanners, which are commonly employed on
flying systems, unsuitable for such environments. Outdoor
environments have many natural features and good lighting
conditions, which makes them well suited for visual odometry
systems. In such environments, previous camera images can be
easily recognized, so the camera can be used for loop closure.
In our approach, we use both laser and stereo odometry
for pose estimation. The combination of two odometry
approaches allows compensating drawbacks of a single sensor. Moreover, the estimation of all six degrees of freedom
(6 DoF) states can be done using only one filter. This differs
from other approaches, where either laser odometry [6] or
monocular visual odometry [7] is used for pose estimation.
The stereo camera in our system points downward not
only to ensure that the odometry is available in outdoor areas
but also to enable detection of a target from above. Drift
errors can be compensated by using key frames in the visual
odometry system, as well as recognition of known landmarks
in a topological map. For the indoor exploration mission, the
map is fixed and predefined, as known landmarks include the
window, door, and chimney. Their exact position is known
with respect to the house, so they can be used to correct drift
errors. These are detected and tracked using front-facing and
upward-facing cameras (not shown in Figure 1), respectively.
Two separate cameras provide more stable tracking than a
pan-tilt unit with one camera of the same weight.
Odometry
Laser Odometry
The laser odometry system is based on Censi's canonical
scan matcher [12]. The laser scan is projected to the ground

plane in the laser transform component, using attitude information from the data fusion component (Figure 2). The projected data are only valid for scan matching if the scanned
environment objects contain vertical planes. This assumption is valid for most indoor environments. The algorithm
uses an iterative closest point (ICP) variant to compute
three-dimensional (3-D) delta movement information
[change in (x, y) position and yaw angle] between two points
in time and the corresponding measurement covariance.
Visual Odometry
A correlation-based algorithm [13], [14] is used to obtain a
disparity image from two time-synchronized camera
images in the stereo-processing component. Based on this
3-D information, the six-dimensional delta position and
orientation between two points in time as well as the corresponding measurement covariance are calculated [15]. The
algorithm supports a key frame buffer so that the delta
measurement refers not just to the last acquired image but
also to the image in the buffer that gives the delta measurement with the smallest absolute covariance.
As shown in the "Experimental Results" section, the
estimated variances for laser and camera odometry are a
good indicator to classify the environment into indoor and
outdoor. In the variance calculation for each sensor, it is
assumed that there are no outliers in the measurement.
During the experiments, we have found that, under bad
sensor conditions, outliers in the measurements occurred.
These could not be detected by an outlier rejection mechanism using Mahalanobis distance. Therefore, the measurement variance is invalid. Fusing these measurements
would lead to unpredictable behavior of the filter. Because
of this, we switch to the sensor that works well in a specific
environment. We assume that the sensor with the smallest
measurement variance is best suited in the current environment and is therefore used for fusion.
Data Fusion
The proprioceptive sensor information from the IMU and
the exteroceptive odometry information have to be fused
to get the current system state estimate. There are two
main challenges.
First, the odometry data give only relative position and
orientation information. Second, the odometry data are time
delayed because of measurement and data processing time.
Precise times of measurement are obtained through hardware synchronization triggers. The total delay of the laser
odometry in the experimental system is about 100 ms with
an update frequency of 10 Hz, and for the visual odometry,
the delay is more than 300 ms, with a frequency of 3 Hz.
Therefore, the measurement refers to a state in the past. As
the estimate is used to control the UAV, and the quadrotor
dynamics are fast compared to the measurement delays, the
latter have to be considered in the data fusion algorithm.
This is realized using an indirect feedback Kalman filter with
state augmentation [16] using two state vectors.
SEPTEMBER 2012

IEEE ROBOTICS & AUTOMATION MAGAZINE

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2012