IEEE Consumer Electronics Magazine - September 2018 - 21

shown in Figure 3). Lidar is becoming very popular in autonomous vehicles. Several prototype vehicles [8], [9] have demonstrated the advantages of using lidar in autonomous driving.
Lidar is useful for systems implementing automatic braking,
object detection, collision avoidance, and more. Depending on
the type of sensor, lidars for cars can have a range of up to
60 m. Despite the aforementioned advantages, lidars are
heavy, bulky in size, and expensive. Moreover, atmospheric
conditions such as rain or fog can impact the coverage and
accuracy of these systems. Emerging solid-state lidars [10]
have opened the possibility of powerful lidars that are significantly smaller and relatively inexpensive.

RADAR
Radar systems emit microwaves and estimate the speed and
distance of an object by measuring the change in the frequency of the reflected wave as per the Doppler effect. Due to the
longer wavelength of microwaves, they can travel much farther than optical light (e.g., with lidar) and can detect objects
at a longer distance. Unlike lidar, radar is not affected by
foggy or rainy weather conditions and is relatively inexpensive. Depending on their operating distance range, radar systems can be classified as short range (0.2-30 m), medium
range (30-80 m), or long range (80-200 m) [11]. Cross-traffic
alerts and blind-spot detection are some of the applications of
short-/medium-range radars. These systems are often located
at the corners of a vehicle. Adaptive cruise control is a longrange radar application, with the system located behind the
front grill or under the bumper. Researchers have been developing algorithms to improve the performance of radar and
reliability all while attempting to reduce the cost and power of
the system [6].

COMPUTER VISION DATA FLOW FOR ADASs
Figure 4 shows the steps involved in a vision-based system,
each of which is discussed.

IMAGE ACQUISITION
This refers to the process of capturing a frame from a video.
The frame is often represented as a matrix of pixel data where
each frame contains three channels of information, e.g., red,
green, and blue (RGB) sets of pixels. Typical frame rates in
ADASs range from five frames per second (fps) to 60 fps
depending on the application. Applications that involve detection of vehicle proximity need a higher frame rate due to the
rapid change in distance for cars on the road. In contrast, traffic sign detection does not demand a higher frame rate
because only one frame of the sign needs to be captured for
the sign to be detected.

PREPROCESSING
There are several common preprocessing steps needed to prepare an image for various computer vision algorithms, e.g.,
denoising, color enhancement, color space conversion, and
image stabilization. A typical example of color space conversion is to convert the RGB color space to hue, saturation, and
value to separate color from the intensity. Moreover, the hue
channel is often used to separate out adverse effects (e.g.,
shadows, uneven lighting, and over- and underexposure) in the
image to make tracking and detection easier.

SEGMENTATION
This is the process of separating features from a frame. In
analyzing an image, it is helpful to partition it into recognizable objects, e.g., identifying the road and sky in a frame as

ULTRASONIC SENSORS
Ultrasonic sensors use sound waves to measure the distance
to an object. These sensors are mainly used for detecting
objects very close to the vehicle. Some example applications
include automatic parking and parallel parking assist. These
sensors are mainly located under the front and rear bumper of
the vehicle.

OTHERS
A few other sensors are used to complement and improve the
functionalities of those discussed earlier. For instance, photonic mixer device (PMD) cameras consist of an array of
smart sensors that enable fast optical sensing and demodulation of incoherent light signals simultaneously [12]. PMDs
can support parallel target pixel-wise distance measurement
without scanning, thus resulting in faster imaging, high lateral resolution, and depth information. IMUs and GPSs are
examples of systems that help improve the distance measurements with lidar and radar.

VISION-BASED ADASs
Vision-based ADASs rely on images from cameras and use
computer vision principles to extract useful information.

FIGURE 3. The Google self-driving-car-generated 3-D image of its
surroundings using lidar [9].

Image
Acquisition

Preprocessing

Segmentation

Object Detection
and Tracking

Depth
Estimation

System Control

FIGURE 4. The vision data flow for the ADAS used.

september 2018

IEEE Consumer Electronics Magazine

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - September 2018