Signal Processing - September 2016 - 64

optical system. The f-number is an inverse measure: i.e., larger
f-numbers correspond to smaller solid angles. For in-camera
light field systems, the f-number of the main lens must always
be larger than that of the microlenses to ensure that light is not
leaking into a neighboring microcamera. At the same time, for
a good directional sampling, the f-number should be as small as
possible. Ideally, the main lens f-number would remain constant
throughout all operational conditions. This requirement imposes additional constraints, especially on zoom systems [21].
The discussion so far has involved ideal first-order optics. In
reality, however, optical systems exhibit aberrations, i.e., deviations from perfect behavior. Initial investigations [22] have
shown that the phase space sampling patterns are deformed
by the main lens aberrations. In addition to the classic distinction between geometric and blurring aberrations, an interpretation of the phase space distortions suggests that directional
shifts (i.e., a directional variant of the geometric distortions)
and directional blur (i.e., a mixture of subview information) are
introduced by aberrated main lenses. The effects of microlens
aberrations are relatively minor and only concern the exact
shape of the sampling kernel.
An example of the distortions introduced by an aberrated
main lens, as opposed to an ideal thin lens, is illustrated in
Figure 5. The horizontal shifts in the sampling patterns correspond to geometric distortion, typically treated by radial
distortion models [18], [23]. The (slight) vertical shifts correspond to a directional deformation of the light field subviews.
A known shifting pattern can be used to digitally compensate
for main lens aberrations [22] or even to exploit the effect for
improving light field sampling schemes [24] (see the "Computational Processing" section).

Calibration and preprocessing
Calibration and preprocessing are tightly interlinked topics for
light field imaging. As outlined in the previous section, many
parameters of a light field camera change when the focus of
the main lens is changed. This concerns not only the geometric
characteristics of the views but also their radiometric properties. The preprocessing of light field images needs to be adapted to account for these changes. In addition, different hardware
architectures require adapted preprocessing procedures. We
will, therefore, cover these steps based on only one example
(here, a Lytro camera, which is an afocal lenslet-based light
field imaging system); the underlying issues, however, affect
all types of in-camera light field systems.

Color demosaicking
Using a standard Bayer color filter array to enable colored light
field imaging appears to be a straightforward choice. However,
as shown in Figure 6(a), for the case of an afocal light field
camera each micro-image encodes the (s, t) dimensions of the
light field. Different color channels, therefore, correspond to
different (s, t) sampling patterns. The final image quality can
be improved by taking this fact into account [25].

Vignetting
The intensity fall-off toward the sides of the micro-images,
also known as vignetting, changes with the optical settings of

Phase Space in World Space

Phase Space in World Space
s

0.1

0.08

0.06

0.04

0.02

0.02
A.U.

A.U.

While a satisfactory treatment of first-order light field
imaging can be achieved by trigonometric reasoning or updated matrix optics techniques, a complete theory of light field
aberrations is missing as of this writing.

-0.02

-0.04

-0.06

-0.08
-0.1
-0.5

-0.08

Thin Lens
0
A.U.
(a)

Double
Gauss f /4

Raytrace

-0.1
-0.5

0.5
u

0
A.U.
(b)

0.5
u

Figure 5. The effect of lens aberrations for an f/4 afocal light field system: (a) the phase space distribution of the sampling pattern in the world space,
assuming an ideal main lens (thin lens), and (b) the phase space distribution of the sampling pattern in the world space using an f/4 double Gaussian
system as a main lens. The sampling pattern is significantly distorted. The highlighted phase space regions correspond to the space between the left and
the right plots. The side subview (purple) is more severely affected compared to the center subview (blue).
64

IEEE SIgnal ProcESSIng MagazInE

September 2016

Table of Contents for the Digital Edition of Signal Processing - September 2016