Signal Processing - September 2016 - 21

filter pixels to do the job. This requires exploiting the latest
The devices discussed above lie in the micro to millimeter
efforts in nano-optics, such as from [22], to use spectrally
scales and are passive in the sense that the coded apertures
selective filters at the desired scales. Another goal is to find
do not change over time and there is no controlled illumiways to exploit low-power programmable optical templates
nation projected onto the scene. This is in contrast to vision
that use technologies such as eInk, which powers many
and graphics methods that use designed lighting to decode
e-readers and which remains static until sufficient energy is
scene information and create new displays. Researchers have
available for a pattern change.
recently began to ask how these methods could work on minAnother potential opportunity is the integration of comiature platforms. For example, a challenge on small devices is
putational photography techniques with existing robotics and
the inherent reduction in baseline. Reference [28] has shown
SLAM techniques for flying microrobots [24], floating sensors
how a circular setup can address some of these challenges for
and surveillance drones. These tools could
photometric stereo. Another direction to
allow, for example, photometric stereo of
address the baseline issue is to move from
Miniature computational
large tourism sites or disaster zones by using
triangulation to time-of-flight using active
varying illumination from multiple drones.
illumination. On the macro-scale, time-ofphotography has great
Temporal visual information at small
flight research has allowed the extraction of
potential for applications
scales
can enable navigation, obstacle
novel scene properties [25]. For miniature
in a variety of fields
avoidance and optical flow; yet processing
systems, trading off the modulated sourcwhere small, networked
video on low-power platforms is prohibies's power consumption versus the depth
platforms are already
tive. CentEye (http://www.centeye.com/)
sensing becomes important.
making an appearance,
has shown embedded computing based
One way to balance these needs and
optical flow at high rates and at low resoenable illumination-based sensing on small
such as agriculture,
lution using embedded vision cameras.
devices would be to extract a signal out of
security, health, and the
Integrating data from multiple sensors has
low wattage illumination. A new generation
internet of Things.
enabled optical flow at real-time rates. For
of computational illumination methods take
extremely fast sampling, it may be possiadvantage of low-power microelectromeble to exploit graded index lenses or optical fibers that can
chanical systems (MEMS) mirrors that have been created for
bend light in curves. Such optical elements can introduce
mobile hand-held projectors, such as those manufactured by
time delay by guiding incoming scene radiance into optiMicrovision, Syndiant, and Cremotech. For example, using a
cal loops, which can be tightly wound in a small volume,
5-W hand-held projector from Microvision, the authors of [21]
enabling, perhaps, fast capture of near simultaneous photohave enabled computational illumination techniques in outgraphs without clocking at extremely high rates.
door scenes, in the face of full sunlight. For miniature compuFinally, since true efficiency is only possible by having
tational photography, the converse is clear; if there is no strong
the sensing task at hand influence every part of the sensor,
ambient illumination, then the same system can be made to
a fascinating question is how to distribute the work load
work at orders of magnitude lower power budgets, since simiover these different components. Should we sample and
lar techniques of exposure synchronization and epipolar rectiprocess with the optics, in such a way as to minimize the
fication can be harnessed to decrease power consumption.
computational load? Or should we use a neuromorphic senWhile these methods prove promising, an interesting
sor to process the measurements as they are made? This
direction put forth by [30] is to engineer a wide-angle MEMS
suggests that design tools in the form of a compiler, to
mirror modulator for enabling futuristic applications such as
allow automatic partitioning of the computing problem into
micro light detection and ranging (LIDAR) by demonstrating
components that can be performed best by optics, coded
an electrothermal MEMS working in liquid for the first time.
sampling, on-board processing, or general-purpose signal
By submerging the MEMS mirror into a mineral oil whose
processing and vision algorithms.
refractive index is 1.47, a wide-angle optical scan (2 120c) was
(1
10 V), and the scan freachieved at small driving voltage
quency reached up to 30 Hz. The power consumption shown
Toward full systems:
was 11.7 mW per degree in the mineral oil.
Societal, legal, and cultural impact
We anticipate a future with trillions of networked miniature
cameras. These computational cameras will be small, cheap,
The next opportunities
numerous, and capable of recovering more information about
Figure 3 depicts shaded gray regions that show the potential
the world around them than today's conventional point-andfor further advances in efficiency and performance. For
shoot cameras. The hypothetical impact of such devices has
example, very few existing techniques take advantage of, say,
been discussed in many contexts, such as within the camera
computing in ASICs at the sensor level and many rely on
sensor network research community, and not all impacts may
conventional PC-based postimage capture processing. Task
be desirable. For example, if these tiny sensors are not biodespecific sampling may also reduce on-board processing; for
gradable, then the potential environmental impact may dwarf
example, a low-power face detector may have an optimal
current concerns on e-waste. Another issue is privacy, as
combination of thermal pixels, polarized pixels and skin
IEEE SIgnal ProcESSIng MagazInE

September 2016

http://www.centeye.com/

Table of Contents for the Digital Edition of Signal Processing - September 2016