Signal Processing - September 2016 - 33

64
32
16
8

13
Simulation

Experiment

27 cm Calibration

128

PSNR (dB)

11
10
9

8
4
4 8 16 32 64 128 256
Scene Distance (cm)
(a)
27 cm
7 cm

7 cm Calibration

Calibration Distance (cm)

256

(b)

(c)

Figure 8. The 3-D imaging with FlatCam. (a) A heat map of the reconstruction PSNR of a simulation of the scene as a function of the scene distance and
the calibration distance of the measurement matrices. At closer scene distances, the reconstruction is sensitive to the choice of multiplexing matrix at the
correct calibration depth; at further scene distances, the sensitivity decreases. (b) An experimental setup showing "FLAT" and "CAM" at different distances
from the camera. (c) Reconstruction at 7 cm and 27 cm through simulation and our prototype FlatCam. The word "FLAT" placed at 7 cm is in focus when
reconstructed using measurement matrices calibrated to depth of 7cm. The word "CAM" placed at 27 cm is in focus when reconstructed using measurement matrices calibrated to depth of 27 cm.

model (5) and commensurately reduce performance. To invesFigure 8(a) shows a heat map of reconstruction PSNR of
tigate the effect of scene motion, we repeated the above experia simulated 2-D scene as a function of the scene distance
ment with three different dynamic scenes that diagonally
and the calibration distance of the measurement matrices.
translate the virtual high-resolution scene X
We see that the reconstruction qualby 0.05, 0.1, and 0.2 pixels per acquisition.
ity improves as the calibration depth of
The lensless imaging
For the slow-moving scene, the reconthe camera approaches the actual scene
approach promises to
struction performance improves with the
depth. Moreover, the sensitivity of the
challenge the traditional
number of acquisitions, just as with a static
reconstruction due to the discrepancy in
barriers of size, weight,
scene. But for the faster-moving scenes, after
these depths decreases with increasing
cost, and performance
peaking early on, the reconstruction perscene distance.
formance deteriorates with the number
Figure 8(c), shows the reconstruction of
in a broad range of
of acquisitions, due to the increased deviaa
3-D
scene at two different depth planes.
applications spanning
tion from the model. Figure 7(c) shows the
For a particular fixed mask, we calibrated
consumer, medical,
reconstructed images using one, nine, and
the measurement matrices with a screen
scientific imaging,
36 measurements for the static scene and the
at the distances of 7 cm and 27 cm. We
machine vision, and
scene moving at 0.2 pixels per acquisition.
accounted for the field of view of the senremote sensing.
The trade-off between spatial and temsor by adjusting the size of the calibration
poral resolution could be improved by
patterns in accordance with the CRA of the
estimating the motion between frames and registering the
sensor. The resulting two sets of matrices were then used to
measurements before reconstructing the image (a difficult, but
create focused images at 7 cm and 27 cm from a single acquisolvable problem; see [33]). Adaptive measurement schemes
sition with FlatCam. (The line artifacts in the experimental
also hold promise for balancing this tradeoff.
reconstruction are due to scene illumination leaking into the
sensor from the sides that was not accounted for in the calibration procedure. We can reduce the unaccounted light in future
3-D imaging
prototypes by introducing baffles.)
FlatCam can computationally change its focus to new depths
in a scene from a single acquisition. The key is that, for a given
mask design, we can calibrate a set of separable measurement
Limitations and challenges facing lensless imaging
matrices {U L i} i = 1,.., L and{U R j} j = 1,.., R , each obtained using a
The very first cameras were lensless (pinhole cameras), but
screen at a different depth (recall Figure 4).
the advent of lenses and other advanced optics relegated such
IEEE SIgnal ProcESSIng MagazInE

September 2016

Table of Contents for the Digital Edition of Signal Processing - September 2016