IEEE Robotics & Automation Magazine - December 2018 - 18

sharing the same temporal and spatial frames. We preferred
this scheme versus a monolithic approach because it provides
higher modularity, flexibility, and efficiency and offers the
capacity to guide the map's development, leveraging the contri-
bution of each sensor.
Figure 6(c) shows the multisensor map obtained in the
experiment in Figure 6(b). The robot localization computed
by the multisensor MCL in that experiment had a mean
error with respect to the real-time kinematic global positioning
system (GPS) of 9.9 cm, which was 47% lower than the
robot localization estimation obtained using lidar odome-
try and mapping in real time [29].
Robot pose estimation during manipulation requires
higher levels of accuracy. To that end, the points and lines
simultaneous localization and mapping (SLAM) algorithm
was developed to leverage the state of the art in SLAM [30],
with simultaneous estimation of point and line features [31]
as well as precise localization at a rate of 3 Hz, resulting in

(a)

(b)

(c)

(d)

better accuracy than direct methods. Parameterizing lines
by their end points provides robustness against occlusions
and in poorly textured environments. To improve localiza-
tion precision, a new method using camera images and
deep learning was used, reaching a precision of 4.5 cm. Fig-
ure 7(a) shows the deep-learning architecture. Faster
odometry estimates are obtained by fusing IMU and optical
flow data. In AEROARMS, sensor synchronization is
addressed with time stamps, enabling timing errors of some
milliseconds, sufficient to obtain the required GNSS-free
pose estimation errors.
Once the pipe is localized, industrial use cases require the
identification and tracking of specific I&M characteristics
and artifacts on the pipe, such as welding marks and corro-
sion points. The specific characteristics to be tracked can
be chosen online by an operator. Offline learning methods
are not feasible because of the small size of the training data
set. We developed an online semisupervised boosting meth-
od [32] that adapts to real-time changes in illu-
mination, shadows, and partial occlusions. In
this case, the operator selects online the type of
defect, and the perception system adapts the
pattern shape, color, and texture to detect and
track the defect.
This new method can work in two modes.
In the first, the system in a sense learns online;
if the defect characteristics change significant-
ly, the system asks the operator to incorporate
these new characteristics in the model. In the
second mode, the system works only by identi-
fying or tracking the defect; although this
method allows some adaptation, major chang-
es are not permitted. An example of weld
detection and tracking using the method is
shown in Figure 6(d).
The deployment of a robot crawler is also a
use case, as mentioned previously. The pickup
and release of the crawler by the aerial robot
are performed in two steps. In the approach
maneuver, the crawler pose is roughly esti-
mated with an appearance-based deep-learn-
ing method, the architecture of which is
shown in Figure 7(b); for the actual pickup
or release, a positioning system based on
deformable markers was built [33], guarantee-
ing very high accuracy (2-cm precision), as
shown in Figure 6(e).

(e)

Figure 6. (a) An aerial vehicle with a multisensor setup; (b) the vehicle aloft
during mapping experiments; (c) a multisensor map built with lidar (orange
dots), ultrawideband (green dots), and stereo vision; (d) pipe weld tracking;
and (e) the precise localization of a robot crawler using artificial markers.

18

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

december 2018

Motion Planning and Navigation
Motion planning is required 1) to move the
aerial manipulator from a takeoff position to
the area to be inspected or maintained and 2)
to perform the inspection or maintenance
task. The former may require navigation in
environments cluttered with pipes, structures,
and so forth (see Figure 2). The latter requires
IEEE Robotics & Automation Magazine - December 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2018
