Signal Processing - September 2016 - 20

Size of Optics
-6

10

10-3 m

m

Power Consumption

100 W or More

10 W or More

Single Viewpoint Image

Full Light Field

Diffraction
Gratings [10] Flatcam [2]
(2015)
(2015)

ASP-Based
Light-Field
Camera [15] (2014)

10-2 m

Partial Light Field
(Depth)

Micro-Baseline
Phot. Stereo [28]
(2015)

Wide-Angle
MEMS
Modulators [30]
(2015)

Episcan [21]
(2015)

No Light Field

On-Sensor Processing Can
Reduce Post-Capture Load

Reduce Power Consumption
with Efficient Illumination

Wide-Angle Sensors [17]
(2013)

Multiple Networked Sensors
Share Load
0.001 W
Goal: Miniature Computational Cameras at Micron Sizes and mW Peak Power (Currently Not Demonstrated)

Figure 3. The first wave of miniature computational cameras. We organize the new wave of small computational cameras according to optical size and
power consumption of the full system. Light-field cameras require powerful on-board computations, but the size of the optics and coded apertures has
reached micron scales. On-board computation at millimeter scales has been proposed for vision sensors, but these do not capture the entire light-field.
We illustrate the new broad steps such as applying sensor-based processing to reduce footprints, applying optical processing to share in the computational load and exploiting efficient active lighting to reduce on-board power consumption. [All images used with permission: [15], [17], [28], and [30]
courtesy of the IEEE; [10] courtesy of Rambus/OSA; and [21] courtesy of ACM.]

multiple hours of PC-grade processing of
The key lesson learned by diffraction is embraced, unlike much of conthe measurements to allow full light-field
these early computational ventional computational photography, which
relies on a ray geometric model of light, albeanalysis. We picked the full power footphotography researchers
it partially augmented with color and polarprint required to generate the key result in
was that important
ization. For example, [13] have shown the
each research paper.
scientific questions
promise of adding micron-scale fabricated
A significant portion of this first wave
polarizing filters to CMOS/CCD cameras.
of miniature computational photography
involved the coded
Exploiting diffraction does not happen as
has been in the realm of lensless imaging,
aperture patterns and
in the optical processing community, where
which has long been valued due its simplicity,
the related decoding
coherent or partially coherent models are
throughput, and potential for miniaturization.
algorithms for images
used to obtain closed form solutions. Instead,
Recent novel image sensor designs recover
captured under
to handle fully incoherent light from the real
angular information for light-field analysis
these apertures.
world, the relative effects of diffraction are
[15]. Reference [10] also used lensless difused to infer scene properties. For example,
fraction patterns to capture angular variations
in [15], angle sensitivity is obtained from the relative effects of a
in the light field. Lensless imaging has played an important role
double decker layer of diffraction patterns. Another idea among
in new types of compressive imagers [2]. Reference [17] demonthese pioneering designs is the use of nonconventional optics
strated an angular theory of wide-angle optical processing and
and coded apertures fused together as one unit. For example, in
showed results for fiducial detection on small, autonomous
[17], optical templates for detecting targets are embedded in a
robots, without needing to capture the entire light field.
refractive slab, enabling the Snell's window effect, and allowing
Certain common ideas are shared among these first few
an extremely wide FOV without using fish-eye lenses.
forays into computational photography in the small. First,
20

IEEE SIgnal ProcESSIng MagazInE

|

September 2016

|
Table of Contents for the Digital Edition of Signal Processing - September 2016
