Signal Processing - September 2016 - 141

constraints to enable eye tracking in more general scenarios.
directed at the same scene object, but, reasoned by the physical
Cazzato et al. derive pupil information and head pose at the
disparity of the eyes, the difference of the per-eye gaze vecsame time to enable gaze tracking while both the eye-tracking
tors can be quite large, e.g., when one is looking at an object
camera and the tracked person can move independently [5].
close-by. This vergence motion allows humans to perceive
Most image and video content, on the
depth in nature and in stereoscopic images.
other hand, is being viewed in quite a preThe unconsciously triggered tracking reflex
While gaze-contingent
dictable fashion. Our visual attention, i.e.,
when a moving object attracts gaze is called
display approaches have
our gaze, is subconsciously drawn to certain
smooth-pursuit eye motion. On a screen,
been proposed before,
image or video regions while other parts of
smooth-pursuit eye motion may affect the
an image are merely being glanced over [6].
acuity function, assuming that the frame
only recently have
In the past two decades, numerous algorender duration is above the retinal integraeye-tracking hardware,
rithms
have been devised to predict visual
tion time. This brightness-level dependent
saliency estimation
saliency of local image features. In 1998,
value describes the average duration of
methods, and graphics
Itti et al. proposed a visual attention estimacollecting photons before transmitting the
hardware become
tor inspired by the early vision processes of
visual information to the higher levels of
sufficiently fast, robust
our visual cortex [7]. Based on a set of prethe visual system [2].
Accomodation is another form of eye
and affordable to allow for defined center-surround operations and linear filters, image features are computed on
motion and describes the mechanical abilincorporating advanced
multiple image resolution scales and comity of the eye to compress or relax the lens
gaze-aware methods in
bined into one saliency map, from which a
of the eye so that objects in the foveal spotmass-market devices.
neural net selects and ranks the most promilight are projected clearly. In other words,
nent features. The predictor of Itti et al. is
accomodation describes the natural counstill competitive to other state-of-the-art methods today.
terpart of adjusting a camera lens so that an object in the scene
Over the ensuing years numerous improvements, as well as
gets into focus. In-focus objects can be described by a depthalternative approaches, to visual saliency prediction have been
of-field plane within the focus distance to the eye and are
proposed [8]. One of the most successful current methods is
clearly projected onto the retina, whereby out-of-focus objects
based on image decompositing and contrast measures akin to
appear blurred.
high-dimensional Gaussian filtering [9]. The resulting saliency
Adaptation is the time-dependent process that adjusts our
map is claimed to successfully separate fore- and background
eye to the environmental lighting situation. Although adapinformation on a per-pixel basis. Alternatively, visual saliency
tation enables us to see over seven orders of magnitude of
may be learned directly from large amounts of eye-tracking
brightness intensities, we are not able to see equally well at
data [10]. Benchmark tests of various computational models of
all intensity levels. The visual acuity for details and color
visual saliency indicate, however, that there is no single methperception is reduced with dimming of light. In other words
od equally suitable for all types of scenes and situations [11].
acuity is lower at scotopic levels (cones) of illumination than
In comparison to work on image saliency, there exists sigat photopic levels (rods). It is, therefore, difficult to read a
nificantly less research into video saliency, although motion and
book in a dim room without additional light. During daytime,
moving objects are known to be strong attractors of visual attencontrast sensitivity is lower, but sharp vision and color vision
tion. Little research has also been invested so far into saliency
work very well.
prediction in the outer peripheral field of view, i.e., into the kind
of visual stimulus that is able to elicit long-range saccades.
Gaze detection and prediction
Because local image saliency determines to a large extent
Gaze-contingent approaches require knowledge of where on
where people look, innumerable painters throughout the centhe screen the user is looking at any time. Depending on the
turies have experimented with visually salient cues to actively
application, gaze direction must be known to about one degree
guide the eye of the beholder. On today's gaze-contingent disaccuracy and updated with at least 50 Hz, while total latency
plays, gaze can be actively steered around using imperceptible
may not exceed 60 milliseconds [3]. In modern eye tracking
local contrast modulations in the peripheral field of view [12].
systems, video cameras are frequently mounted either close to
By turning the modulation stimulus off immediately after the
the eyes or at a distance to record the pupils and corneal refleconset of the elicited saccade, saccadic masking prevents the
tions. If the user is allowed to move the head, it, too, needs to
user from being consciously aware of the stimulus. The modube tracked. The cameras must be calibrated and the video
lation stimulus is never perceived by the user's foveal vision.
streams processed in real time to continuously determine gaze
direction. A comprehensive guide to eye tracking methods has
recently been compiled by Holmqvist et al. [4].
Gaze-contingent techniques
Active eye-tracking methods require additional hardware
The notion of gaze-contingent display devices dates back at
and calibration of the hardware for each user. The calibration
least two decades. Excellent earlier review articles on gazeprocess may hamper deployment of gaze-contingent applicontingent techniques and applications include those of Reincations. Just recently, researchers began to loosen calibration
gold [13] and Duchowski [14].
IEEE SIgnal ProcESSIng MagazInE

September 2016

141

Table of Contents for the Digital Edition of Signal Processing - September 2016