Signal Processing - September 2016 - 57

framework. Almost all ToF systems can be characterized by
the model parameters " p, h, { , . Depending on the problem
at hand, the role of " p, h, { , and the associated algorithms
needs to be adapted. For example, the SRF discussed in context of MPI appears naturally in other problems, such as
ultrasound tomography [29], single photo imaging [31] and
light detection and ranging (LiDAR) [30]. However, the
probing function and the IRF are very different for each case.
In particular, consider the case of LiDAR. The probing function is modeled as p = d and the IRF is a parametric function
of form { ^ t h = ae^a k - T0ht + b k , where " a k, b k ,4k = 1 take four different values with continuous transitions, depending whether
t ! {I k} 4k = 1 , where I k is an instrument or sensor dependent
quantity. This gives rise to a new form of sampling kernel
r (TD-ToF case). Hence,
z = { [32], as opposed to z = p * p
we believe that by systematically studying the role of
" p, h, { , across various ToF problems-optical and nonoptical-better insights may developed.
Another interesting direction may be to consider the case
when the SRF is modeled by a differential equation. For example, in the case of fluorescence lifetime imaging, the associated differential equation is L m = 2 t + ^1/mh , and the resulting
reflected signal is the solution to CL m 6r ^ t h@ = p ^ t h, t $ 0 .
The SRF in this case is the Green's function. Similar ideas
may be used to develop algorithms for imaging through scattering/diffusive media where L is some differential equation
that models diffusion. The parameters of L encode physical
properties such as lifetime or scattering coefficient.

Probing function
Since the probing function is the only available degree-of-freedom in ToF imaging pipeline, it is important to understand what
mathematical principles should be used for designing probing
functions. Waveform design is a known art in radar and wireless
communications. However, such options are rarely considered in
optical ToF systems. Maximum length sequences for TD-ToF
and sinusoids for the FD-ToF are the de facto examples. On the
hand, it may not always be feasible to calibrate the probing function. In that case, it may be worthwhile to use blind deconvolution algorithms for image reconstruction.

Algorithms and fundamental limits
MPI is a significant problem in ToF imaging and a number of
papers have attempted to address this issue-both in time- and in
frequency domain. However, to date, most of the results remain
empirical and rarely discuss any details on performance guarantees. In a recent work [24], we used Cramér-Rao bounds in context of TD-ToF-based multiple depth imaging. For instance, for
the TD-ToF case, one may write the probing function as a Fourier series with Fourier coefficients {z m} m ! Z . In this case, two
optical paths, Dd apart, are resolvable provided that
Dd $ c
T
4r

1
PSNR

S,
N

where T = 2r/~ 0 is the fundamental time period, PSNR is
the peak-signal-to-noise-ratio, S -1 = / m ! Z m 2 | z m | 2 and N

is the number of measurements. For parametric SRFs, we
believe that more such efforts could lead to hints on interesting applications of ToF sensors and motivate new problems in
sampling theory. This could be the key to questions such as:
when can two lifetimes in fluorescence lifetime imaging
setup be super-resolved?

Modeling nonideal reflections
In our experience, the SRF of form C k d ^t - t k h [c. f. (6)] only
approximately models a reflection. In a practical setting,
C k } k ^t - t k h may serve as a good starting point for modeling
reflections. Here, } k is a filter that models the interaction of
the probing function with the material property or accounts
for distortion, system nonlinearities and dispersive media. In
seismic engineering, terahertz spectroscopy, and ultrasound
systems, this behavior is much more pronounced as material
properties play an important role when the probing function
undergoes a reflection.

FD-ToF
This is an interesting mode of operation since most consumer
ToF systems are based on FD-ToF, which uses phase estimation. As seen in (4), if i ~ 2 2r , the depth estimates suffer
with ambiguity or phase-wrapping problem. Previous solutions use coprime frequencies [33], however, there is room for
improvement. For example, in theory, phase is a linear function of frequency, i ~ = 2~d/c . This is not the case in practice
and leads to erroneous depth estimates with multiple frequency measurements. Hence, a desirable phase estimation
algorithm should jointly correct for any distortions and
phase-wrapping.
(4)
Calling m ^t, ~h = C 0 ^1 + (p 20 /2) cos ^~t + i ~hh, i ~ is estimated by sampling in time-domain, that is, m k = m ^kT, ~ 0 h
given a fixed modulation frequency ~ 0 . Alternatively, one
may use multifrequency sampling using m k = m ^t 0, k~ 0h
for the estimation of i ~ . This gives rise to a broader question of when can time-frequency sampling be used, that,
m ,,k = m ^,t 0, k~ 0 h in context of solving inverse-problems
linked with ToF imaging. Specifically, when ~ = t for depth
imaging, the problem boils down to parameter estimation of
chirp signals. Finally, in view of (8), phase retrieval algorithms
may be designed when only intensity measurements are available, that is, m k 2 = m ^t 0, k~ 0h 2 .

Sensor design for higher modulation frequencies
Most consumer-grade ToF sensors are based on continuous
wave probing functions. Currently, such sensors work with high
fidelity up to a modulation frequency of about 80 MHz. We
believe that much of the interesting physical phenomenon may
only be observed as higher frequencies. For example, higher
modulation frequencies will certainly enhance depth resolution and MPI correction capabilities. In context of fluorescence lifetime imaging [21], shorter lifetimes may be
resolved with higher modulation frequencies. Similarly, subsurface scattering properties can be studies with streak tubes
[10]. This hints that higher modulation frequencies are the

IEEE SIgnal ProcESSIng MagazInE

|

September 2016

|

57



Table of Contents for the Digital Edition of Signal Processing - September 2016

Signal Processing - September 2016 - Cover1
Signal Processing - September 2016 - Cover2
Signal Processing - September 2016 - 1
Signal Processing - September 2016 - 2
Signal Processing - September 2016 - 3
Signal Processing - September 2016 - 4
Signal Processing - September 2016 - 5
Signal Processing - September 2016 - 6
Signal Processing - September 2016 - 7
Signal Processing - September 2016 - 8
Signal Processing - September 2016 - 9
Signal Processing - September 2016 - 10
Signal Processing - September 2016 - 11
Signal Processing - September 2016 - 12
Signal Processing - September 2016 - 13
Signal Processing - September 2016 - 14
Signal Processing - September 2016 - 15
Signal Processing - September 2016 - 16
Signal Processing - September 2016 - 17
Signal Processing - September 2016 - 18
Signal Processing - September 2016 - 19
Signal Processing - September 2016 - 20
Signal Processing - September 2016 - 21
Signal Processing - September 2016 - 22
Signal Processing - September 2016 - 23
Signal Processing - September 2016 - 24
Signal Processing - September 2016 - 25
Signal Processing - September 2016 - 26
Signal Processing - September 2016 - 27
Signal Processing - September 2016 - 28
Signal Processing - September 2016 - 29
Signal Processing - September 2016 - 30
Signal Processing - September 2016 - 31
Signal Processing - September 2016 - 32
Signal Processing - September 2016 - 33
Signal Processing - September 2016 - 34
Signal Processing - September 2016 - 35
Signal Processing - September 2016 - 36
Signal Processing - September 2016 - 37
Signal Processing - September 2016 - 38
Signal Processing - September 2016 - 39
Signal Processing - September 2016 - 40
Signal Processing - September 2016 - 41
Signal Processing - September 2016 - 42
Signal Processing - September 2016 - 43
Signal Processing - September 2016 - 44
Signal Processing - September 2016 - 45
Signal Processing - September 2016 - 46
Signal Processing - September 2016 - 47
Signal Processing - September 2016 - 48
Signal Processing - September 2016 - 49
Signal Processing - September 2016 - 50
Signal Processing - September 2016 - 51
Signal Processing - September 2016 - 52
Signal Processing - September 2016 - 53
Signal Processing - September 2016 - 54
Signal Processing - September 2016 - 55
Signal Processing - September 2016 - 56
Signal Processing - September 2016 - 57
Signal Processing - September 2016 - 58
Signal Processing - September 2016 - 59
Signal Processing - September 2016 - 60
Signal Processing - September 2016 - 61
Signal Processing - September 2016 - 62
Signal Processing - September 2016 - 63
Signal Processing - September 2016 - 64
Signal Processing - September 2016 - 65
Signal Processing - September 2016 - 66
Signal Processing - September 2016 - 67
Signal Processing - September 2016 - 68
Signal Processing - September 2016 - 69
Signal Processing - September 2016 - 70
Signal Processing - September 2016 - 71
Signal Processing - September 2016 - 72
Signal Processing - September 2016 - 73
Signal Processing - September 2016 - 74
Signal Processing - September 2016 - 75
Signal Processing - September 2016 - 76
Signal Processing - September 2016 - 77
Signal Processing - September 2016 - 78
Signal Processing - September 2016 - 79
Signal Processing - September 2016 - 80
Signal Processing - September 2016 - 81
Signal Processing - September 2016 - 82
Signal Processing - September 2016 - 83
Signal Processing - September 2016 - 84
Signal Processing - September 2016 - 85
Signal Processing - September 2016 - 86
Signal Processing - September 2016 - 87
Signal Processing - September 2016 - 88
Signal Processing - September 2016 - 89
Signal Processing - September 2016 - 90
Signal Processing - September 2016 - 91
Signal Processing - September 2016 - 92
Signal Processing - September 2016 - 93
Signal Processing - September 2016 - 94
Signal Processing - September 2016 - 95
Signal Processing - September 2016 - 96
Signal Processing - September 2016 - 97
Signal Processing - September 2016 - 98
Signal Processing - September 2016 - 99
Signal Processing - September 2016 - 100
Signal Processing - September 2016 - 101
Signal Processing - September 2016 - 102
Signal Processing - September 2016 - 103
Signal Processing - September 2016 - 104
Signal Processing - September 2016 - 105
Signal Processing - September 2016 - 106
Signal Processing - September 2016 - 107
Signal Processing - September 2016 - 108
Signal Processing - September 2016 - 109
Signal Processing - September 2016 - 110
Signal Processing - September 2016 - 111
Signal Processing - September 2016 - 112
Signal Processing - September 2016 - 113
Signal Processing - September 2016 - 114
Signal Processing - September 2016 - 115
Signal Processing - September 2016 - 116
Signal Processing - September 2016 - 117
Signal Processing - September 2016 - 118
Signal Processing - September 2016 - 119
Signal Processing - September 2016 - 120
Signal Processing - September 2016 - 121
Signal Processing - September 2016 - 122
Signal Processing - September 2016 - 123
Signal Processing - September 2016 - 124
Signal Processing - September 2016 - 125
Signal Processing - September 2016 - 126
Signal Processing - September 2016 - 127
Signal Processing - September 2016 - 128
Signal Processing - September 2016 - 129
Signal Processing - September 2016 - 130
Signal Processing - September 2016 - 131
Signal Processing - September 2016 - 132
Signal Processing - September 2016 - 133
Signal Processing - September 2016 - 134
Signal Processing - September 2016 - 135
Signal Processing - September 2016 - 136
Signal Processing - September 2016 - 137
Signal Processing - September 2016 - 138
Signal Processing - September 2016 - 139
Signal Processing - September 2016 - 140
Signal Processing - September 2016 - 141
Signal Processing - September 2016 - 142
Signal Processing - September 2016 - 143
Signal Processing - September 2016 - 144
Signal Processing - September 2016 - 145
Signal Processing - September 2016 - 146
Signal Processing - September 2016 - 147
Signal Processing - September 2016 - 148
Signal Processing - September 2016 - 149
Signal Processing - September 2016 - 150
Signal Processing - September 2016 - 151
Signal Processing - September 2016 - 152
Signal Processing - September 2016 - 153
Signal Processing - September 2016 - 154
Signal Processing - September 2016 - 155
Signal Processing - September 2016 - 156
Signal Processing - September 2016 - 157
Signal Processing - September 2016 - 158
Signal Processing - September 2016 - 159
Signal Processing - September 2016 - 160
Signal Processing - September 2016 - 161
Signal Processing - September 2016 - 162
Signal Processing - September 2016 - 163
Signal Processing - September 2016 - 164
Signal Processing - September 2016 - 165
Signal Processing - September 2016 - 166
Signal Processing - September 2016 - 167
Signal Processing - September 2016 - 168
Signal Processing - September 2016 - 169
Signal Processing - September 2016 - 170
Signal Processing - September 2016 - 171
Signal Processing - September 2016 - 172
Signal Processing - September 2016 - 173
Signal Processing - September 2016 - 174
Signal Processing - September 2016 - 175
Signal Processing - September 2016 - 176
Signal Processing - September 2016 - Cover3
Signal Processing - September 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com