Signal Processing - September 2016 - 140

Table 1. An overview of common display device types.
Display Type

Refresh Rate

Vertical Resolution

Pixels per inch

Lossless Viewing
Distance

Lossless FOV

Smartphone

60 Hz

1,920p/5.5 in-3,840p/5.5 in

400.5-801.0

> 8.5-4.3 in

< 32-64°

Tablet

60 Hz

1,200p/7 in-1,824p/12.3 in

323.5-267.0

> 10.6-12.9 in

< 20-30.4°

Monitor

60-120 Hz

1,080p/23 in-2,880p/27 in

123.1-217.6

> 35.9-15.8 in

< 18-48°

TV set

60-600 Hz

1,080p/55 in-4,320p/85 in

40.0-103.6

> 85.8-33.2 in

< 18-72°

Projector

24-1,000 Hz

1,080p/125 in-2,160p/125 in

17.6-35.2

> 195.1-97.5 in

< 18-36°

HMD

60-90 Hz

800p/7 in-1,200p/7 in

215.6-352.9

> 15.9-9.79 in

< 13.3-20°

Each column contains values for a middle-class device and a high-end device. Perceptually lossless minimum viewing distance and maximum lossless field of view (FOV) are given
for a person with average vision (20/20 Snellen) so that the perceived resolution matches foveal acuity.

readability of small-scale text and playback of high-resolution
videos. This implies, however, that more and more pixels must
be rendered on resource-strapped mobile devices. Incongruities also exist in the temporal domain between digital video
recording and display capabilities. While TV display refresh
rates today commonly match or exceed 60 Hz, standard video
acquisition frame rates still hover between 24 and 30 frames/
second (fps). The discrepancies between the physical world
and its digital representation, as well as the mismatch in acquisition versus display capabilities and displays versus human
visual perception, lead to noticeable artifacts.
If it is known, or can be reliably estimated, how human
vision perceives digital images at any one time, gaze-contingent display methods are able to make use of a number of
perceptual strategies to improve perceived visual quality. In
addition, gaze contingency allows allocating computational
resources on the fly to image regions that are perceptually
relevant for the current gaze direction. While gaze-contingent
display approaches have been proposed before, only recently
have eye-tracking hardware, saliency estimation methods, and
graphics hardware become sufficiently fast, robust and affordable to allow for incorporating advanced gaze-aware methods
in mass-market devices. This article highlights recent examples of gaze-contingent computational display approaches that
enhance perceived visual fidelity of common, consumer-market display technologies.

Modeling human vision
Gaze-contingent displays exploit abilities and limitations of
the human visual system (HVS). Although correlations
between the eye and the visual cortex are not yet fully understood, models of the HVS enable us to conservatively express
some important features of human vision.
Visual acuity provides an estimate of the smallest visual
detail that the HVS is spatially able to perceive. The acuity
follows approximately the distribution of cones and rods in
the retina. It reaches its highest value, therefore, in the foveal
region, which can be modeled by a central disc with a radius of
two degrees visual angle. Acuity falls off rapidly with eccentricity in the periphery. Therefore, visual acuity is estimated by
140

a function over the eccentricity or, in other words, the distance
to the fovea given in degrees visual angle.
The acuity value is commonly given either as the Snellen
value or a minimum angle of resolution (MAR) value [1]. For
healthy young adults, the common highest visual acuity is
defined as 20/20 Snellen or, equivalently, 1 minute of arc in
terms of MAR for the foveal region [1]. The acuity limit is primarily reasoned by the spacing of photoreceptors in the retina.
However, due to the large variability between human eyes,
estimating the distribution of rods and cones across the retina
is difficult. Additionally, studies have shown that, at eccentricities greater than two degrees, the acuity is worse than what
can be predicted from cone spacing [1]. Consequently, visual
acuity cannot be determined from the distribution of the photoreceptors on the retina only.
The psychophysical model of Aubert and Foerster from
1857 is a well-established model for low-level vision tasks [1].
It states that the minimum discernible angular size increases
roughly linearly with eccentricity for the first 20-30c; it then
rises more rapidly [1]. Due to its simplicity and conservative
approximation of the real acuity, the model of Aubert and
Foerster is still commonly used. However, the linear model of
Aubert and Foerster does not do justice to the full complexity
of the HVS, as peripheral vision is not a scaled-down version
of the foveal area [1]. Eye adaptations in very bright and dark
areas as well as motions of the eye also influence the amount of
detail perceived. Therefore, sophisticated acuity models must
include means to deal with additional vision features. Color
vision is another aspect affected by the distribution of rods
and cones. Although color can be still perceived in nonfoveal
vision if the stimulus is large enough, retinal performance falls
off linearly until 20-30c in periphery for color discrimination
and many other visual tasks as well [1].
Vision models for gaze-contingent displays may not only
consider visual acuity, brighness adaptation, and color vision
but also the measureable movements of the human eyeball. The
most important motion abilities of the eye are saccades, the
motion when jumping from one object of interest to another,
and fixations, which give humans the ability to directly gaze
intentionally into a certain direction. Both eyes are commonly

IEEE SIgnal ProcESSIng MagazInE

September 2016

Table of Contents for the Digital Edition of Signal Processing - September 2016