Tech Briefs Magazine - May 2023 - PIT-3

Circuit Filter Image sensor
Board
Filter
3 cm
5 μm
(a)
small portion of the incoming light split
by dispersive optics; roughly speaking, the
sensitivity is inversely proportional to the
number of spectral bands (colors). Such
limited performance results in poor usability
and may be the reason why HS imaging
is not yet widespread.
Concept of Compressed SensingBased
HS Imaging
To break through the physical limits of
sensitivity, compressed sensing (CS) is
applied to HS imaging. CS is a technique
to efficiently acquire and reconstruct signals
from under-sampled measurements
and has been applied in several fields
such as medical inspection and deep
space exploration. An example application
in the field of optics includes a miniaturized
spectroscopic device, overcoming
the physical limitations of device size
with CS. Here, CS is used to efficiently
acquire the spatial and spectral 3D information
of HS image using a 2D image
sensor, enabling high sensitivity HS imaging.
Compared with the conventional HS
imaging approaches that detect one color
per pixel, the developed method detects
multiple colors per pixel. Furthermore,
by randomly modulating the light
intensity for each wavelength at each pixel,
thereby spectrally and spatially encoding
light, the spatial and spectral 3D information
can be reconstructed from the
captured 2D image. The effective sensitivity
is then determined by the average
transmittance resulting from the random
modulation. The effective sensitivity ideally
becomes 50 percent regardless of the
number of spectral bands, as the average
of 0-100 percent random modulation.
To correctly reconstruct the HS image
from the captured 2D image, the incomPhotonics
& Imaging Technology, May 2023
(b)
ing light must be randomly modulated
in the spatial and spectral dimensions,
i.e., spatially and spectrally random sampling.
Spatial randomness can be evaluated
from the histogram of transmission
patterns at each wavelength, and spectral
randomness can be evaluated by calculating
the 2D correlation coefficient
between transmission patterns (rij) at
different wavelengths. Quantitatively,
the reconstruction error due to the spatial
randomness is less than 2 percent,
same level as shot-noise, when the standard
deviation (σ) divided by the average
transmittance (μ) is higher than 0.1.
For reconstruction errors due to spectral
image
sensor
(Spatial)
Scanning
Type
(c)
(a) A photograph of an image sensor with a special filter. (b) An optical microscope image of the special filter. (c) A schematic image of light detection with
thinning out using the special filter. (Image: Panasonic)
randomness, rij should be less than 0.9
to keep the reconstruction error below 2
percent.
Even if our CS-based approach is theoretically
possible, the challenge is how to
achieve both spatially and spectrally random
light transmission in the system. In
the present case, it was achieved by a
coded mask designed as a random array
of Fabry-Pérot filters. By optimizing the
structure of the Fabry-Pérot filter at each
pixel, the light transmission can be properly
designed to obtain the required randomness.
Considering the complexity of
the fabrication process and the randomness
of the light transmission, 64 differImage
sensor
convolution
image
sensor
Snapshot
Type
optical
component
bandpass
filter
shooting
area
shooting area
y
λ
x
×
object
measurement matrix
capture
reconstruction
compressed image
CS-Based Type
A comparison of conventional methods and compressed sensing-based methods for hyperspectral
imaging. (Image: Panasonic)
3
HS image
T
λ
scan

Tech Briefs Magazine - May 2023

Table of Contents for the Digital Edition of Tech Briefs Magazine - May 2023