Signal Processing - November 2016 - 40

Head-Mounted
Eye-Tracker

EEG
Headband

ECG Sensors
Dash-Mounted
Eye-Tracker

Respiration Belt

(a)

(b)

FIGURE 4. Example research setup at the University of Toronto for (a) a driving simulator and (b) an instrumented vehicle. The simulator shown is a fixedbased simulator, paired with a dashboard-mounted eye tracker. Both the driving simulator and the instrumented vehicle are set up to provide common
vehicle measures, including speed, lane position, brake/accelerator pedal position, and headway distance. In the instrumented vehicle, the driver is shown
wearing a head-mounted eye tracker. Electrocardiogram (ECG), galvanic skin response (GSR), and respiration sensors are also attached to driver's clothing to illustrate potential sensor placement on the participant. For actual data collection, physiological sensors are to be placed directly on the skin.

with an NIR source were widely used to exploit the special
for driver state monitoring. A commercial example is the
optical property of human pupils known as the dark/bright
Lexus Driver Monitoring System, which was implemented
pupil effect. When the illumination is coaxial with gaze path,
in some Lexus vehicles beginning in 2006 (e.g., Model GS
pupils reflect light back to the source and appear bright in
450h) and in some Toyota vehicles beginning in 2008 (e.g.,
the acquired images, while an offset of illumination placeCrown Hybrid), and which utilizes six built-in near-infrared
ment would not create such an effect and would result in dark
(NIR) light-emitting diode (LED) sensors as well as a chargepupils [22]. For this system, pupils are located by simple subcoupled device camera facing toward the driver. It performs
traction of the alternating dark pupil frame and the bright
eye tracking, eyelid detection, and head motion detection to
pupil frame, a method that is computationally cheap [22].
detect the onset of sleepiness from facial expressions and
Although this eye-tracking technique is accurate in the labowarns the driver.
ratory environment, its heavy reliance on the hardware setup
A typical system for analyzing facial and body expresdegrades its robustness for realistic driving scenarios. Based
sions is illustrated in Figure 5. First, visual input is acquired
on advancements in computational power and machineusing one or more cameras. A camera system with an active
learning algorithms, it is possible to develop intelligent comlighting source (NIR) is often used to alleviate the problem
puter vision solutions with large video data input for real-time
of extremely large illumination variations during driving.
applications that do not rely heavily on image acquisition
Then, the region of interest or facial/body landmarks need to
hardware configurations. For example, the
be located by either direct detection from
boosted cascade classifier, the underlying
each frame, or with the help of tracking
Observable cues from
idea behind the Viola-Jones object recogalgorithms. At the next stage, metafeadriver's facial and body
nition algorithm [23], has been incorpotures, such as PERCLOS, head pose, and
expressions can be used
rated in many recent camera-based driver
gaze direction (and different statistics
to infer about a driver's
face or eye detection algorithms (e.g., see
associated with them), are extracted. Finalarousal level.
[24]). Furthermore, tracking algorithms
ly, the system makes a decision by looking
are commonly utilized for enhancing the
for particular patterns in the time series
processing speed of the overall system. These algorithms
of the metafeatures, which often involves fusion with other
allow us to replace the costly processing of a full frame of
signals as will be discussed later in the "Data Fusion" secthe input image by the lower cost searching for the target in a
tion. A comprehensive review of associated algorithms and
more constrained manner. Algorithms such as Kalman filter
techniques used in the literature is provided in [11] and [21].
or particle filter have been commonly used in both eye trackThe first fundamental task for this system is to successers with structured lighting [25] and computer vision-based
fully locate desired tracking features in each frame in real
systems [24].
time, which can be the face, eyes, or more precise facial landNext, discriminant metafeatures can be calculated based
marks. The ensuing software detection algorithm is highly
on the obtained lower-level features. This step can be seen as
dependent on the image acquisition hardware used. In earlier
higher-level feature extraction. Many of the visual indicators
studies, structured illumination/camera eye-tracking systems
40

IEEE SIgnal ProcESSIng MagazInE

November 2016

Table of Contents for the Digital Edition of Signal Processing - November 2016