Signal Processing - September 2016 - 52

Scene Response Function
Time Domain

h2(t )

Constituent Sinusoids

∧
zω = h2∗(ω)

Fourier Domain

Γ1

t1

t2

Phasor Domain

Γ1e jωt1

Γ2

(a)

Multiple Frequency Measurements

ω
t

zω1

Γ2e jωt2

Γ1e jωt1

FD-ToF Measurements

(b)

Amplitude (a.u.)

1

Measured
Estimated

0.5

Γ2e jωt2

0
-0.5
-1
10 15 20 25 30 35 40 45 50
Harmonic Frequencies (Steps of 0.7937 MHz)

zω2
Constituent Sinusoids

Figure 4. Multipath interference in ToF sensors. (a) The scene response function for K = 2 in time-domain [cf. (6)], frequency-domain, and complex

phasor-domain. At a given modulation frequency ~ , the ToF sensor measures the complex number z ~ . However, the correct values to be estimated are
" C k , d k ,Kk =- 01 . This model mismatch results in erroneous depth reconstruction. (b) Experimentally measured multifrequency data and its decomposition
into sinusoids via spectral estimation.

capability of ToF sensors in Figure 5, where up to K = 3
optical paths interfere at the sensor. As seen in Figure 5, the
resulting 3-D images for various modulation frequencies
are erroneous. Although Prony's method for spectrum
estimation is highly unstable in the presence of noise and
model mismatch, oversampling-that is, using much more than
2K measurements-is a reasonable solution to counteract perturbations in data. Again, we use the matrix pencil method to
estimate the multiple depth related scene parameters. With 46
multiple frequency measurements [cf. Figure 4(b)], the estimation procedure shows that it is possible to reconstruct objects at
multiple depths even in presence of MPI. The recovered amplitude and phase images, " C k ,2k = 0 and " t k ,2k = 0 , respectively, are
shown in Figure 5.

Transient imaging and sampling sparse signals
Understanding light propagation through physical medium has
interesting theoretical and practical consequences. For example, a recent demonstration by Velten and coworkers [10]
showed that the information contained in multiple echoes of
light can be used for non-line-of-sight imaging. From a signal
processing perspective, the properties of light/wave propagation can be best understood as a time-dependent transfer function of the scene. The three main ingredients of time-resolved
light transport are direct reflections, inter-reflections and subsurface scattering:
C 0 d ^t - t 0h
144
42443
Direct Reflection

52

/ k = 1 C k d ^t - t k h
14444
244443
K

Interreflections

C s d ^t - t s h ) e - nt u ^ t h,
1444442444443
Subsurface Scattering

where " C k, t k ,Kk =-01 and {C s, t s} are the usual multipath components, μ is the absorption coefficient, and u ^ t h is the Heaviside
function. If it were possible to create a highly time-localized
probing function of form p ~ d , identification of scene related
parameters, "" C k, t k ,Kk =-01, C s, t s, n , would be a relatively easy
task. However, the time-resolution of probing functions programmed on the state-of-the-art ToF sensors is, at best, about
11 ns [cf. Figure 3(a)]. This is orders of magnitude longer than the
sophisticated apparatus used by Velten et al. [10], which attains a
pulse width of approximately 50 ps. That said, TD-ToF sensors
may still be used for imaging transient phenomenon linked with
direct and indirect inter-reflections. In fact, this problem is closely tied with the problem of sampling of continuous-time sparse
signals [27].
The SRF for the transient imaging problem with direct and
indirect components of the light propagation is a continuous-time
K sparse signal,
(6 )

h K ^ t h = C 0 d ^t - t 0h + / k = 1 C k d ^t - t k h .
144
42443 14444244443
K

Direct Reflection

Indirect Reflections

With the IRF defined in (1), the ToF sensor measurements are
m ^ t h = ^r 7 p h ^ t h = ^ p * pr * hr K h ^ t h (cf. "ToF Image Formation Model."). With the commutativity property of the
convolutions, the sampled TD-ToF measurements may be
written as m n = ^z * hr K h ^ t h t = nT , n = 0, f, N - 1 , where
r h ^ t h. Since the probing function admits a bandz^ t h = ^ p * p
limited approximation [see (3) and Figure 3(a)], the time-reversed
TD-ToF measurements may be reinterpreted as samples of a

IEEE SIgnal ProcESSIng MagazInE

|

September 2016

|
Table of Contents for the Digital Edition of Signal Processing - September 2016
