Signal Processing - September 2016 - 51
in (2), the reflected signal reads r ^ t h = Cp ^t - t 0h, t 0 = 2d 0 /c .
Again, the lock-in sensor acts as an electronic homodyne detector such that (1) holds, and measurements results in [25],
(S2)
m^ t h = C 0 c 1 +
p 20
cos ^~t + i ~hm, i ~ ! 60, 2rh,
2
hK ^ t h =
(4)
where i ~ = ~t 0 = 2d 0 ~/c is the frequency dependent phase.
The ToF sensor records discrete measurements of form,
m k = m ^kT h with T = r/2~ and uses a phase estimation algorithm commonly known as the "Four Bucket Method" [22], [21]
to estimate parameters d 0 and C 0 . For a given modulation frequency, this method works with four discrete measurements
" m k ,3k = 0 that are used to form a complex number z ~ ! C ,
z ~ = (m 0 - m 2) + . (m 3 - m 1) with
m0 m1
2 + p 20 cos (~t 0) 2 - p 20 sin (~t 0)
;
E = C0 =
G.
2 2 - p 20 cos (~t 0) 2 + p 20 sin (~t 0)
m2 m3
A consequence of multiple optical paths combining at a given
pixel is that the shift-invariant SRF for single depth (2) now takes
the form of a K-sparse filter,
(5)
u0 =
and
The scene parameters are then estimated by C
ud 0 = c+z ~ /2~ . We use this method to create the depth image
in Figure 1. The raw data samples corresponding to the
experiment are plotted in Figure 3(b). This completes our discussion on depth imaging with time- and frequency-domain
ToF sensors.
Key takeaways of this section
■
■
■
The ToF imaging pipeline consists of a probing function,
scene response function, and an instrument response function.
For the case of 3-D imaging, the scene response function is
the shift-invariant, time-delay operator in (2).
In TD-ToF, the probing function is a time-localized pulse [cf.
Figure 3(a)], and the object's distance from the sensor is
encoded as the round-trip time-delay t 0 = 2d 0 /c.
In FD-ToF, the probing function is a sinusoidal waveform
with known modulation frequency [cf. Figure 3(b)]. The
object's distance from the sensor is encoded as a frequency
dependent phase.
Landscape of inverse problems in ToF imaging
Having covered the mathematical basics of time- and
frequency-domain ToF imaging systems, we will now discuss
case studies where signal processing theory may be used in
combination with ToF sensors to solve inverse problems.
Optical multipath interference and spectral estimation
Multipath interference (MPI) is a problem that naturally
occurs in communications, acoustics, and array signal processing. Consumer-grade FD-ToF sensors are designed to
work under the premise that each point in the scene maps to a
single pixel on the sensor. Whenever this hypothesis does not
hold, the ToF sensor measurements are erroneous due to MPI.
This results in corrupted 3-D images. For example, this can
happen when imaging through a semitransparent object, such
as a glass window.
tk = 2 dk ,
c
/ C k d^t - t kh,
k=0
(6)
where K is the number of optical paths, and " C k, d k ,Kk =-01 are the
scene parameters corresponding to each optical path. For simplicity, we assume that p 0 = 1 in the definition of the probing
function, p ^ t h = 1 + p 0 cos ^~t h . The reflected signal in this
case reads
rK ^ t h =
K-1
K-1
k=0
k=0
/ C k cos ^~t - ~t kh = / C k cos ^~t - i k,
~
h
(7)
which indicates an addition of sinusoids with varying phases.
Let us resort to complex-valued representation and let z * denote
the complex-conjugate of z. The measurements take the form
m K (t) = 1 e .~t
2
z ~ /p 20
■
K-1
K-1
/ Ck e+.
~t k
=0
1k44
4t2
4443
)
= 1 e .~t ht )K (~)
2
h K (~)
K-1
with ht K (~) = / C k e - .~t k ,
k=0
1444442
444443
(8)
Fourier Transform
where ht K ^~h is the Fourier transform of the shift-invariant
SRF in (6)-a sum of K complex exponentials or phasors.
Since the probing function is a sinusoid-an eigenfunction of a
linear system-the measurements amount to observing Fourier
transform at modulation frequency ~ .
Whenever K = 1-the case of single-depth 3-D imaging with no MPI-the unknown constant ht K ^~h is directly
estimated by implementing the four-bucket method that maps
" m K ^rk/2~h,3k = 0 " z ~ = ht *1 ^~h. However, in the presence
of MPI, the FD-ToF sensor estimates the scalar z ~ = ht *K ^~h
rather than " C k, d k ,Kk =-01 -the pixel-wise scene parameters. For
K = 2 , we plot the SRF in time-domain, frequency-domain,
and phasor-domain in Figure 4(a). The fact that z ~ = ht *K ^~h
is an argument of modulation frequency motivates the necessity for frequency diversity [25]. Given N harmonic measurements of form " z n~ 0 ,nN=-01 , the task of estimating 2K unknowns
" C k, d k ,Kk =-01 is a classical problem in signal processing that is
studied under the theme of spectral estimation theory [26]. Due
to the pervasiveness and wide applicability of this problem, a
self-contained review of Prony's method that seeks to estimate
" C k, d k ,Kk =-01 given " z n~ 0 ,nN=-01 is provided in "Spectral Estimation
(Prony's Method)." This approach may be used for the correction
of MPI in ToF sensors. In Figure 4(b), we show multifrequency
data acquired using a ToF sensor with K = 3. Our demonstration, which is based on the matrix pencil method [26], shows
the constituent sinusoidal components. Other methods such as
the orthogonal matching pursuit [25], EPIRIT/MUSIC [26], or
atomic norm thresholding [34] may also be used.
While MPI suggests that interfering optical paths are a nuisance, it may be exploited for 3-D scanning and imaging of
translucent objects. We demonstrate multiple depth imaging
IEEE SIgnal ProcESSIng MagazInE
|
September 2016
|
51
Table of Contents for the Digital Edition of Signal Processing - September 2016
Signal Processing - September 2016 - Cover1
Signal Processing - September 2016 - Cover2
Signal Processing - September 2016 - 1
Signal Processing - September 2016 - 2
Signal Processing - September 2016 - 3
Signal Processing - September 2016 - 4
Signal Processing - September 2016 - 5
Signal Processing - September 2016 - 6
Signal Processing - September 2016 - 7
Signal Processing - September 2016 - 8
Signal Processing - September 2016 - 9
Signal Processing - September 2016 - 10
Signal Processing - September 2016 - 11
Signal Processing - September 2016 - 12
Signal Processing - September 2016 - 13
Signal Processing - September 2016 - 14
Signal Processing - September 2016 - 15
Signal Processing - September 2016 - 16
Signal Processing - September 2016 - 17
Signal Processing - September 2016 - 18
Signal Processing - September 2016 - 19
Signal Processing - September 2016 - 20
Signal Processing - September 2016 - 21
Signal Processing - September 2016 - 22
Signal Processing - September 2016 - 23
Signal Processing - September 2016 - 24
Signal Processing - September 2016 - 25
Signal Processing - September 2016 - 26
Signal Processing - September 2016 - 27
Signal Processing - September 2016 - 28
Signal Processing - September 2016 - 29
Signal Processing - September 2016 - 30
Signal Processing - September 2016 - 31
Signal Processing - September 2016 - 32
Signal Processing - September 2016 - 33
Signal Processing - September 2016 - 34
Signal Processing - September 2016 - 35
Signal Processing - September 2016 - 36
Signal Processing - September 2016 - 37
Signal Processing - September 2016 - 38
Signal Processing - September 2016 - 39
Signal Processing - September 2016 - 40
Signal Processing - September 2016 - 41
Signal Processing - September 2016 - 42
Signal Processing - September 2016 - 43
Signal Processing - September 2016 - 44
Signal Processing - September 2016 - 45
Signal Processing - September 2016 - 46
Signal Processing - September 2016 - 47
Signal Processing - September 2016 - 48
Signal Processing - September 2016 - 49
Signal Processing - September 2016 - 50
Signal Processing - September 2016 - 51
Signal Processing - September 2016 - 52
Signal Processing - September 2016 - 53
Signal Processing - September 2016 - 54
Signal Processing - September 2016 - 55
Signal Processing - September 2016 - 56
Signal Processing - September 2016 - 57
Signal Processing - September 2016 - 58
Signal Processing - September 2016 - 59
Signal Processing - September 2016 - 60
Signal Processing - September 2016 - 61
Signal Processing - September 2016 - 62
Signal Processing - September 2016 - 63
Signal Processing - September 2016 - 64
Signal Processing - September 2016 - 65
Signal Processing - September 2016 - 66
Signal Processing - September 2016 - 67
Signal Processing - September 2016 - 68
Signal Processing - September 2016 - 69
Signal Processing - September 2016 - 70
Signal Processing - September 2016 - 71
Signal Processing - September 2016 - 72
Signal Processing - September 2016 - 73
Signal Processing - September 2016 - 74
Signal Processing - September 2016 - 75
Signal Processing - September 2016 - 76
Signal Processing - September 2016 - 77
Signal Processing - September 2016 - 78
Signal Processing - September 2016 - 79
Signal Processing - September 2016 - 80
Signal Processing - September 2016 - 81
Signal Processing - September 2016 - 82
Signal Processing - September 2016 - 83
Signal Processing - September 2016 - 84
Signal Processing - September 2016 - 85
Signal Processing - September 2016 - 86
Signal Processing - September 2016 - 87
Signal Processing - September 2016 - 88
Signal Processing - September 2016 - 89
Signal Processing - September 2016 - 90
Signal Processing - September 2016 - 91
Signal Processing - September 2016 - 92
Signal Processing - September 2016 - 93
Signal Processing - September 2016 - 94
Signal Processing - September 2016 - 95
Signal Processing - September 2016 - 96
Signal Processing - September 2016 - 97
Signal Processing - September 2016 - 98
Signal Processing - September 2016 - 99
Signal Processing - September 2016 - 100
Signal Processing - September 2016 - 101
Signal Processing - September 2016 - 102
Signal Processing - September 2016 - 103
Signal Processing - September 2016 - 104
Signal Processing - September 2016 - 105
Signal Processing - September 2016 - 106
Signal Processing - September 2016 - 107
Signal Processing - September 2016 - 108
Signal Processing - September 2016 - 109
Signal Processing - September 2016 - 110
Signal Processing - September 2016 - 111
Signal Processing - September 2016 - 112
Signal Processing - September 2016 - 113
Signal Processing - September 2016 - 114
Signal Processing - September 2016 - 115
Signal Processing - September 2016 - 116
Signal Processing - September 2016 - 117
Signal Processing - September 2016 - 118
Signal Processing - September 2016 - 119
Signal Processing - September 2016 - 120
Signal Processing - September 2016 - 121
Signal Processing - September 2016 - 122
Signal Processing - September 2016 - 123
Signal Processing - September 2016 - 124
Signal Processing - September 2016 - 125
Signal Processing - September 2016 - 126
Signal Processing - September 2016 - 127
Signal Processing - September 2016 - 128
Signal Processing - September 2016 - 129
Signal Processing - September 2016 - 130
Signal Processing - September 2016 - 131
Signal Processing - September 2016 - 132
Signal Processing - September 2016 - 133
Signal Processing - September 2016 - 134
Signal Processing - September 2016 - 135
Signal Processing - September 2016 - 136
Signal Processing - September 2016 - 137
Signal Processing - September 2016 - 138
Signal Processing - September 2016 - 139
Signal Processing - September 2016 - 140
Signal Processing - September 2016 - 141
Signal Processing - September 2016 - 142
Signal Processing - September 2016 - 143
Signal Processing - September 2016 - 144
Signal Processing - September 2016 - 145
Signal Processing - September 2016 - 146
Signal Processing - September 2016 - 147
Signal Processing - September 2016 - 148
Signal Processing - September 2016 - 149
Signal Processing - September 2016 - 150
Signal Processing - September 2016 - 151
Signal Processing - September 2016 - 152
Signal Processing - September 2016 - 153
Signal Processing - September 2016 - 154
Signal Processing - September 2016 - 155
Signal Processing - September 2016 - 156
Signal Processing - September 2016 - 157
Signal Processing - September 2016 - 158
Signal Processing - September 2016 - 159
Signal Processing - September 2016 - 160
Signal Processing - September 2016 - 161
Signal Processing - September 2016 - 162
Signal Processing - September 2016 - 163
Signal Processing - September 2016 - 164
Signal Processing - September 2016 - 165
Signal Processing - September 2016 - 166
Signal Processing - September 2016 - 167
Signal Processing - September 2016 - 168
Signal Processing - September 2016 - 169
Signal Processing - September 2016 - 170
Signal Processing - September 2016 - 171
Signal Processing - September 2016 - 172
Signal Processing - September 2016 - 173
Signal Processing - September 2016 - 174
Signal Processing - September 2016 - 175
Signal Processing - September 2016 - 176
Signal Processing - September 2016 - Cover3
Signal Processing - September 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com