Signal Processing - September 2016 - 111

Continuous
Linear
Physical

Discrete Physical
Potentially
Nonlinear

EM
Transform

Propagation

Measurement

Discrete
Digital
Algorithmic
Parameter
Estimation

Output

Object
Encoding

Labeling and
Classification

Output

Output

Entrance
Pupil

Figure 3. A schematic representation of an imaging system.

measurement and the digital postprocessing domains. The
opportunities for computational imaging depend upon the
architecture of the system [4]-[13]. In this article, we review
work that we have done for conventional imagers and for pupil,
or Fourier plane, imagers.

Foundations of computational imaging
Figure 3 is a schematic representation of a generic imaging
system. To the left of the entrance pupil is a natural scene
consisting of self-luminous objects or objects illuminated passively, i.e., we do not have active control over the scene illumination. The electromagnetic field incident upon the
entrance pupil exists in three spatial dimensions (x, y, z) and
one temporal dimension t and exhibits intrinsic physical properties of wavelength m and polarization p. The amplitude of
the field is represented by a (x, y, z, t, m, p) .
All elements to the right of the entrance pupil are under
a designer's control and together define the imaging system.
The imager's front end contains elements that manipulate the
incident wave front. The front-end electromagnetic processing
is represented by linear, continuous integral transforms based
on physical models.
After manipulation by the front end, the transformed wave
front impinges on a detector or transducer of some kind.
Transduction is a nonlinear physical process in terms of field
amplitude. It is the irradiance of the field that is transduced,
f = | a | 2 . Also, discrete sampling is implicit in transduction.
The continuous values (x, y, z, t, m, p) over which f is defined
are now discrete.
A matrix representation can be used to provide a mathematical description of the processes up to and including measurement [14], [15]
g = Hf + n,

(1)

where g is the measurement, f is a sampled representation of
the scene irradiance in the object domain, H is the system
transfer or measurement matrix, and n is noise introduced in

the measurement process. The propagation of f from the
object domain to the imager is included in H. Thus, H consists of both natural and engineered components.
A parameter estimator T is applied in postdetection either
to estimate f or some property or parameter of f denoted Xf,
i.e., either
Tf = Tg,

= THf + Tn,

(2)

Xf = Tg,
= THf + Tn.

(3)

or

In contrast to the measurement matrix H, which operates linearly on f, T can be linear or nonlinear.
The last block in Figure 3 represents additional processing
beyond estimation, namely, labeling or classification. That is,
based upon the properties estimated, elements within the scene
are discriminated from one another and assigned to a particular, discrete class of objects. Depending on the application,
classification may not be necessary. We combine classification
processing with estimation into a single transformation T.
It is important to note the special case of T = I , where I is
the identity matrix. This is called a direct measurement, where
the measurements correspond directly to the parameters of
interest. For example, conventional imaging is a case of direct
measurement of scene irradiance values in object space. The
goal in designing a conventional imaging system is to produce
a response that is as close as possible to a d-function over all
expected operating conditions.
Another classic example of direct measurement is optical
matched filtering, e.g., [16]. A matched filter is designed to
detect an object o in the object scene. Its ideal performance is
such that, wherever o is present in the scene f, the measurement
produces a large response at its location. Locations where o is
not present produce a small response. Thus, T outputs the location of o and, by virtue of its design, allows the classification of

IEEE SIgnal ProcESSIng MagazInE

|

September 2016

|

111
Table of Contents for the Digital Edition of Signal Processing - September 2016
