Signal Processing - September 2016 - 56

40 MHz
60

40

40 40

20

20
|zkf0|

60

20

40

60

0
6
5

40

4
3

20

2

∠zkf0
20

40

60

∠z 40f 0 = 5.5822
6
6
4
2
5
4

50

3

60

70

2

80

1

90

6
Phase (Radians)

20 MHz

60

100

30
60 50 40
110
90 80 70
110 100

2

-2

-6
-40

Measured Phase
Estimated Phase
-10
20
Frequency (MHz)

40

1
(a)

(b)

Figure 8. Frequency-domain ToF imaging. (a) Multifrequency measurements of a scene containing a fluorescent sample with a lifetime x = 32 ns. We show
amplitude image | z 20f0 | (in dB) as well as the phase image +z 20f0 (in radians) with f0 = 1 MHz. We also show a 3-D plot of the phase image. This shows the
effect of multipath interference. The phase at the background pixel is 4.1625 rads, which relates to a depth of 2.4842 m, while the actual depth is 2.5 m. At
this depth, the phase recorded as the sample location is 5.5822. The extra 3.3316 is attributed to the fluorescent lifetime decay phenomenon tan -1 ^~m 0h .
(b) At the sample location, we plot phase +z kf0 = - +ht ^kf0h, k = 1, f, 40 (12) recorded by the ToF sensor. We also plot the phase estimated by our method.

available. With t 0 = 0 , estimation of m 0 is a classical problem
in system identification [28]. The presence of nonzero t 0 motivates development of new algorithms. In contrast to [21], where
the authors use nonlinear least squares fitting, here, we develop
a closed form solution to the problem. In view of (12), let us set
y n = z n~ 0 /p 20 and ~ n = n~ 0 , and we have,
yn + 1
1 + .~ n m 0 - .~ 0 t 0
=
e
1 + .~ n + 1 m 0
yn
2
y n + 1 (1 + .~ n + 1 m 0) = y n (1 + .~ n m 0) e - .~ 0 t 0 ,
which is a linear system of equations in m 0 and e - .~ 0 t 0 and can
be solved with any four contiguous values such that
, # n 1 , + 4, , ! Z . We solve this linear system of equations
for " y n ,nn == 20
2 since the modulation frequencies in range
f0 = 2 - 20 MHz are highly stable. While system stability and
signal integrity is one part that affects the SNR, variation of
sample spread on the slide may also lead to weak emission. For
this purpose, we use a confidence threshold for pixel amplitudes.
We compare our system identification approach with previously used nonlinear least squares fitting [21], which uses
N = 40 samples and favorable initialization conditions to obtain
(nanoseconds) mu 0
mu 0, NNLS
(meters)
du
du NNLS

32.16
30.33
2.54
2.494

32.32
30.79
2.49
2.496

31.46
31.51
2.54
2.496 .

For comparable distance estimates, the identification method provides a more robust estimate of lifetimes. In comparison
56

to NNLS fitting (based on MATLAB'S curve-fit toolbox),
our relatively modest and noniterative method is orders of
magnitude faster (~ 10 3 ). We close this subsection by highlighting that the phase/depth images in ToF sensors may be
used for encoding interesting physical/material properties.

Key takeaways of this section
■

■

■

Multipath interference (MPI) in FD-ToF mode can be reinterpreted as a spectrum estimation problem. To recover MPI
components, one must acquire multiple frequency measurements. Our demonstrations were based on the matrix pencil
method [26] but any other method is an option. For example, in [25], the authors use orthogonal matching pursuit.
Transient imaging in TD-ToF mode can be recast as a sparse
sampling problem [27], where the sampling kernel z is the
autocorrelation of the probing function. Whenever the probing
function admits a bandlimited approximation, one may use
spectral estimation to recover the transient image components.
We demonstrated that fluorescence lifetime imaging can be
performed with ToF sensors. We used system identification
methods to estimate an unknown, parametric transfer function linked with lifetime imaging.

Conclusions and future directions
We hope that we have convinced you that ToF sensors are more
than just depth sensors! In what follows, we present our concluding remarks with hints on possible future research directions.

ToF imaging pipeline
We started with an image formation model that allows for
studying different ToF modalities under one common

IEEE SIgnal ProcESSIng MagazInE

|

September 2016

|
Table of Contents for the Digital Edition of Signal Processing - September 2016
