IEEE Robotics & Automation Magazine - June 2020 - 146

270 m
(c)
320 m

(a)

(e)

(b)

(d)

(f)

(g)

Figure 7. A real-word deployment in the Hong Kong International
Airport. (a) and (b) Aerial views of the experiment environment
in Google Maps. (c)-(g) Real autonomous transportation
scenarios.

object detection results, the object-tracking system is of
great importance to ensure safety. Tracking can also help to
improve the real-time performance of the detection system
and provide additional prediction information about
dynamic objects.
* For the laser-tracking module, we match the current
point cloud frame with the tracking template to determine corresponding objects. The laser-tracking template
is generated considering the previous detection result,
tracking result, and estimated object motion information.
The laser-tracking results (LTRs) can also provide
searching-area candidates for the visual-tracking module.
* For the visual-tracking module, the state-of-the-art
approach [19] usually suffers from performance degradation since its tracking template is updated by considering only the previous tracking results. In
industrial applications, tracking results may not
always be accurate; once a fault occurs, the consequent tracking performance cannot be ensured. In
addition, the tracking template should be adjusted
dynamically during the whole task period, and a new
template needs to be generated in the presence of new
objects. To solve this, we modify the approach in [19]
by presenting a new template-updating mechanism.
More specifically, we update the tracking template
with the latest detection result if the following criteria
are satisfied simultaneously: 1) the detected and
tracked objects are judged the same one by recognizing each object's category and computing the intersection over union, and 2) the confidence of the
detection result reaches a sufficiently high level.
146

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2020

Compared with [19], leveraging the feature from
detection results leads to a high leap on both tracking
effectiveness and detection robustness.
● Object motion estimation: Based on the previous tracking
results, an object motion estimation module is presented
using an EKF framework to integrate results from the
laser- and visual-tracking modules and then to estimate
the object's motion state. The vehicle motion state and
environment map are also considered to obtain object
motion estimations in the global coordinate.
The proposed detection and tracking modules can
ensure acceptable performance in most cases; however, failures still cannot be avoided in extreme cases. To solve this,
an ensemble detection module can be developed that acts as
a high-level detector for exhaustively utilizing both the
laser- and visual-based detection and tracking results. Currently, the consistency index is considered in the proposed
perception system to eliminate false detection results, and
the inconsistent results are regarded as potential objects that
are considered only by the motion control system to ensure
safety. In the future, the probabilistic model will be introduced to build the complete ensemble detection module.
Real-Word Deployment and Application
The proposed autonomous vehicle (the Toyota diesel tractor
52-2TD25 with proper self-driving retrofitting) has been
successfully deployed in the cargo terminal of the Hong
Kong International Airport to achieve automated cargo
transportation tasks between the tarmac area and the warehouse. Figure 7(a) and (b) shows the experiment environment in Google Maps; we test the proposed localization,
mapping, and perception approaches within the blue area
(about 80,000 m2) and conduct autonomous cargo transportation tasks within the red area. Transportation routes
are depicted that include both indoor and outdoor routes.
For a single round, the total route length is about 1,250 m.
According to the requirement of administration departments, the upper bound of the vehicle velocity in executing
automated transportation tasks is restricted to 15 km/h; in
addition, two security guards stay ready on the vehicle to
take it over in any emergency.
The proposed system has been tested for more than two
months (the average operation time is about 3.5 h/day). During the long run, the proposed perception system shows
robust performance in different weather and illumination
conditions, and no collision has occurred. The results of the
proposed perception system are shown in Figures 6 and 8. In
Figure 8(a), different colors represent different classes of
objects: green-vehicle, red-airplane/person, purple-container,
and black-trailer. In Figure 8(b), we show three consecutive
frames, where the white blocks are visual detection results
and the colored blocks are visual-tracking results. One sees
that the blue vehicle cannot be detected in frame 1 and the
red vehicle cannot be detected in frames 2 and 3. These missing detection cases show the nonrobustness of the visual
detection approach to small-scale objects in the image under

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020