IEEE Robotics & Automation Magazine - December 2018 - 108

1) algorithms: integration of the building blocks in an independent selfdriving full-stack software, accounting
for sensor handling, data processing,
perception, localization, decision making, and control
2) development tools: off-line software
tools for data handling; a training
framework for artificial intelligence
(AI)-based algorithms, testing, verification, and maintenance
3) hardware: a high-performance, lowpower automotive-grade processor,
allowing low-level optimization of
algorithms, with specialized architectures for NN calculations (inference).
Algorithms
The role of sophisticated algorithms in
solving the three main tasks of self-driving (perception, localization, and planning) is to find the right balance
between traditional and AI-based algorithms. It is widely accepted that AI will
be the driving force behind the software,
although the
specific role of
machine intelliThe role of
gence and the
sophisticated
point of deployment occupy a
algorithms in solving
wide spectrum,
the three main tasks
depending on
the approach
of self-driving is to
used [5]. One
find the right balance of these is the
application of
between traditional
end-to-end
and AI-based
black-box learning by observaalgorithms.
tion; here, the
car learns how
to navigate, negotiate, and adjust control signals solely
by observing human drivers.
Although promising results have
been achieved in simulated environments (for which computer games provide an efficient development platform),
the limited system transparency leads to
difficult fault tracing, and testing the
system for every case is nearly impossible. A more robust but significantly
more complex way is to analyze human
driving and define the building blocks
of a self-driving software. These blocks
108

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

have a hierarchic relation and are
intended to aid the main task by
providing information about the environment, the vehicle-environment
interaction, and an optimal trajectory
to be executed:
● perception: low- and high-level sensor
fusion (information integration),
object detection and classification,
and abstract environment reconstruction
● localization: global localization and
routing, mapping, odometry, and
local positioning
● planning: scenario interpretation,
tracking and prediction, motion
(maneuver) planning, local trajectory
planning, and actuator control.
Development Tools
In the automotive industry supply
chain, car manufacturers (as original
equipment manufacturers) play a significant role in component integration,
building individual and independent
components into the final system.
These are usually provided by first-level automotive suppliers, resulting in a
largely distributed platform in terms of
functionality. Future cars will not have
the luxury of this type of distribution,
because all components will need to
function in harmony, progressing
toward a centralized processing architecture. As a natural consequence, the
simultaneous development of components and their constant cross testing is
becoming an integral part of self-driving
development, where custom off-line
development tools provide a platform
for the following:
● data handling: data collection, annotation (labeling), generation and
enhancement, pre- and postprocessing, and sensor calibration
● algorithm support: a flexible training
environment for AI algorithms and
frameworks for NN inference optimization and high-level sensor
fusion (the association of various
sensor data to real-world instances)
● testing: algorithm verification
(precision, recall, false rejection),
benchmarking and metrics development, and complex off-line
simulation.

december 2018

Processing Hardware
Although the performance of the algorithms is low-bounded by their minimally required reliability factors, the
available processing power poses
another challenge and an upper bound
to complexity, which, in the case of
massive use of AI-based algorithms,
mainly affects the inference of NNs. In
the case of convolutional NNs (CNNs),
one of the most efficient DL approaches, this limits the number and size of
layers, creating a need for optimized
network architectures and relying on
today's vast state-of-the-art networks.
Now, general-purpose computing on
graphics processing units (GPUs) is a
standard way of training and inferencing NNs for sensor data processing and
decision making, taking advantage
of optimized inference engines for
massively parallel GPU computation.
However, GPUs, as general-purpose
computing units, have been primarily
optimized for pixel-by-pixel computations on graphics engines, making
them suboptimal for processing CNNs
per se. This fact has initiated a new era
of hardware development, with chip
design increasingly being focused on
hardware acceleration of NN inference,
increasing the performance density of
these processing units, and, due to their
predicted heavy use in the automotive
industry, allowing automotive safety
integrity-level compliance. This
explains the great interest of chip manufacturers in this domain.
Sensors on Self-Driving Cars
Owing to the large variety of off-theshelf sensors available, there are no
identical prototype platforms among
research communities and industrial/
technological companies. Furthermore,
the choice of sensors affects the hierarchy and fusion of algorithms, and vice
versa: a structured approach will determine the position and type of sensors
placed around the vehicle. Cameras
with different fields of view, lidar units,
radars, and ultrasonic sensors are often
considered essential. Some of these
might play the role of primary sensors,
often completely taking over the role of
other sensors.
IEEE Robotics & Automation Magazine - December 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2018
