IEEE Consumer Electronics Magazine - September 2018 - 22

vision applications include Caffe, Darknet, and MATLAB. An
application of a CNN for object tracking is discussed in [14].
Kalman-filter-based object tracking is proposed in [15], where
the filter tracks the object's velocity.

IMUs and GPSs are examples of
systems that help improve the
distance measurements with
lidar and radar.

DEPTH ESTIMATION

two different features. Various thresholding techniques are
used to filter one class of pixels (e.g., the road) from another
(e.g., the sky). One of the methods, e.g., exploits color information to detect a stop sign, where an algorithm may look for
red in the image (typical for stop signs in the United States).
Any pixels in that red range will be turned white, and anything that is not will be turned black, as shown in Figure 5(a).
This results in a binary image that is often used as a mask for
finding the area of interest on the original image.

OBJECT DETECTION AND TRACKING
This is the process of classifying an object in an image (e.g.,
determining if an object ahead is a vehicle, sign, or pedestrian)
and predicting its movement. It is often accomplished with various machine-learning (ML) algorithms. ML algorithms are
provided large training data sets (thousands of images) to learn
and differentiate between vehicles and common objects found
around them. An example of an object detection method is
called the cascade classifier, which was first presented in [13]
for face detection, on low-performance hardware systems.
Another common technique to train and classify images is
using a convolutional neural network (CNN), which typically
consists of an input layer, multiple hidden layers, and an output
layer. The hidden layers consist of convolution and pooling
layers that are used for feature extraction and a fully connected
layer for classification. Examples of CNN frameworks used for

This step involves estimating the distance of an object in the
image frame relative to the camera. There are two common
methods for depth estimation: 1) the use of a stereo camera to
create a stereo pair and develop them to make a depth map
and 3-D point cloud that allow a real-world reconstruction of
the scene [16]; and 2) the use of a monocular camera and several state-of-the art techniques that use a subset of optical
flow, calibration, and least squares techniques [17].

SySTEM CONTROL
This is the last step in the vision data flow, which involves
interpretation of the outputs from previous layers, as shown
in the vision data flow diagram in Figure 4. This step requires
a weighing of each layer in the vision pipeline to come up
with a confidence value that can be used to make decisions. A
major challenge at this step is a false detection with high confidence that would take priority over other information
obtained from the previous layers. Thus, training with data
that is correct and contains many orientations of the object to
be classified is crucial to achieve high accuracy.

OUTDOOR MONITORING
In this section, we will discuss the classification of objects that
are outside a vehicle, e.g., pedestrians, vehicles, and roads.

PEDESTRIAN DETECTION

Detecting pedestrians is done using various classifiers, e.g.,
[18]. Often more than one classifier is used for detecting people because of the varying orientation
and configuration in which pedestrians
may appear. Deep-learning networks
Confidence = 0.921278 String = STOP
such as CNNs have been helpful to not
only identify pedestrians but also classify their actions.
Confidence = 0.907121 String = STOP

Confidence = 0.889147 String = STOP

(a)

(b)

FIGURE 5. The stop sign detection: (a) a binary stop sign and (b) stop sign classification
using optical character recognition.

22 IEEE Consumer Electronics Magazine

september 2018

VEHICLE DETECTION
Vehicle detection is a major focus of
object detection in ADASs. The fact
that many vehicles share common features, such as having tires, brake lights,
and license plates, allows the detection
of these objects to indicate the presence of a car. These features are all
used to distinguish the vehicle from
other objects, such as signs, roads, and
other miscellaneous objects. In Figure 6,
an example of vehicle detection is shown,
using a CNN framework (Darknet)
and a real-time detection system, You
Only Look Once [19]. The orientation

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - September 2018