Signal Processing - September 2016 - 61

light field [9] if the surface S agrees with some object geometry.
In this case, the directional component of the function describes
the object reflectance convolved with the incident illumination.
Commonly, the additional assumption is made that the surface S
is convex, e.g., by taking the convex hull of the scene. In this case, the
rays can be propagated to other surfaces in the outside region without loss of information. Typically, a plane p is used as the domain
of (parts of) the light field function. The most popular parameterization of the spatial and directional dimensions of the light field is
the two-plane parameterization, which is obtained by propagating
a ray from surface S to the light field plane p (see Figure 1). The
parameterization then consists of the intersection position (u, v)
of the ray with the light field plane p and its intersection with an
additional parallel plane at a unit distance (ut , vt ) . The second intersection is usually parameterized as a difference with respect to the
(u, v) position and called (s = ut - u, t = vt - v) . This second set
of coordinates measures the direction of the ray.

Phase space
The coordinates obtained in this way can be considered as an
abstract space, the so-called "ray phase space" or simply phase
space. A point (u, v, s, t) in this space corresponds to a ray in
the physical space. It is important to remember that the phase
space is always linked to a particular light field plane p. Changing the plane, in general, changes the phase space configuration, which means that a fixed ray will be associated with a
different phase space point.
The phase space is interesting for several reasons. First, it allows
us to think more abstractly about the light field. Second, a reduction to two dimensions (u, s) is easily illustrated and generalizes
well to the full 4-D setting. Third, finite regions of the ray space,
in contrast to infinitesimal points, describe ray bundles. The phase
space is, therefore, a useful tool for visualizing ray bundles. Finally,
an extensive literature exists on phase space optics (see, e.g., [10])
with available extensions to wave optics. The phase space is also a
useful tool for comparing different camera designs [11].
The light field can now be thought of as a radiance-valued
function defined in the phase space, i.e. l (u, v, s, t) , meaning
that each ray, parameterized by (u, v, s, t), is assigned a radiance value l. The task of an acquisition system is to sample and
reconstruct this function.

s
p

s
u
Outside

u
Inside
Phase Space
(b)

Physical Space
(a)

Figure 1. A basic description of light field. (a) The "inside" region contains the scene of interest, while the "outside" region is empty space and
does not affect light propagation. The light field is a function assigning a
radiance value to each of the rays exiting through the boundary surface
\mathscrS . (b) A phase space illustration of the colored rays. A point
in phase space determines a set of ray parameters (u, s) and, therefore,
corresponds to a ray. The phase space is associated with the plane p.
Because the four rays indicated in the subfigure in (a) converge to a
point, the corresponding phase space points lie on a line.

s
p

∆s
u
Outside

u
Inside
Physical Space

Phase Space in p
(a)
s

p
u

∆u

Outside

u
Inside
Physical Space

Phase Space in p
(b)

Figure 2. The finite sampling of a light field with real hardware. (a) Assum-

Light field sampling
The simplest way to sample the light field function is by placing a pinhole aperture into the light field plane p. Were the
pinhole infinitesimal, ray optics a decent model of reality, and
light considerations negligible, we would observe one column
of the light field function at a plane a unit distance from the
light field plane p. In the following, we will refer to that plane
as the sensor plane q. Associating a directional sample spacing of Ds and shifting the pinhole by amounts of Du enable a
sampling of the function, as shown in Figure 2.
A slightly more realistic model is that the directional variation s is acquired by finite-sized pixels with a width equivalent to the directional sample spacing Ds . This introduces a
directional sampling kernel that, in the phase space, can be

ing a sensor placed at the dashed plane and an infinitesimal pinhole results
in a discretization and averaging of only the directional light component. In
phase space, this constitutes a row of vertical segments. (b) A more realistic
scenario uses a finite-sized pinhole, resulting in ray bundles integrated by
the sensor's pixels. In conjunction, pixels and pinholes define a two-aperture
model. In the phase space, the ray bundle passed by two apertures is
represented by a rhomb.

interpreted as a vertical segment, as in Figure 2(a). Of course,
the pinhole has a finite dimension Du , as well. The pinhole/
pixel combination, therefore, passes a bundle of rays, as indicated in Figure 2(b). The phase space representation of the ray
bundle passing this pinhole/pixel pair is a sheared rectangle,
as shown on the right in Figure 2(b). It should be noted that
the pinhole size and the pinhole sample spacing, as well as the

IEEE SIgnal ProcESSIng MagazInE

September 2016

Table of Contents for the Digital Edition of Signal Processing - September 2016