Signal Processing - May 2017 - 62

(a)

(b)

(c)

FIGURE 7. Depth reconstruction results for a scene with one moving object and one static object: (a) a multiframe reconstruction, (b) a single-frame
reconstruction, and (c) a scalable depth sensing.

single-frame and multiframe reconstruction. Accordingly, the
intensity of the speckle is set as
B H (x, y) = 2 L - m max {Pn (x, y)} .
n = 1, 2, 3

(11)

One of the hybrid patterns is shown in Figure 6(c). In this
work, two sets of hybrid patterns with different frequencies
of fringes are used for phase unwrapping when performing
multiframe reconstruction, and patch-based image matching
is adopted for single-frame reconstruction from the speckle as
in the Kinect. To enable region-wise mode transition between
single-frame and multiframe reconstruction, a motion mask that
separates multiple objects with different motions is estimated.
Figure 7 shows an example where there are two objects in the
scene: one is moving and one remains static. As can be seen,
this scalable depth-sensing method adaptively produces lowdelay depth for the moving object and high-accuracy depth
for the static object. In general, single-frame depth reconstruction is computationally intensive due to the correlationbased image matching. For example, it is reported in [31] that
the correlation computation for generating one depth map
takes up to seven seconds on a standard personal computer
with an Intel Core i7-950 processor. Although the acquisition
can be performed at a speed as high as 500 fps in [30], the
reconstruction needs to be conducted offline. Using the central processing unit (CPU) for multiframe reconstruction and
the graphics processing unit (GPU) for single-frame recon-

struction, this work realizes real-time scalable depth sensing
for the first time. The depth reconstruction speed is 20 fps at
640 × 480 pixels.
Since scalable depth sensing well coincides with the characteristics of the human visual system and can be easily realized with low-cost hardware, it provides a practical solution for
many real-world applications when both accuracy and speed
of depth sensing are concerned. In fact, a compromise always
has to be made between accuracy and speed in developing
commodity depth cameras according to the application scenarios, and scalable depth sensing achieves an efficient balance
between these two requirements.

Binary phase shifting

Phase shifting can achieve high-accuracy 3-D measurement,
but it is vulnerable to motion. To alleviate the motion effect,
high-speed hardware is required, but the high-speed DLP projector is expensive. In practice, there is a speed limit to generate 8-b grayscale patterns using a DLP projector. However,
it is much faster to generate 1-b binary patterns, since only
switchings between two states are needed. Moreover, as demonstrated by the Kinect, it is possible to generate binary patterns using a low-cost laser-diffuser emitter. Therefore, it will
greatly enhance the speed or reduce the cost of phase shifting
if binary patterns can be utilized to produce sinusoidal fringes.
An early work along this line is described by Lei and Zhang
in [33]. The basic idea is to use projector defocusing. The defocused projector is modeled by a point
spread function that can be approximated as a Gaussian smoothing filter.
They find that a sinusoidal fringe pattern
can be produced by applying the Gaussian filter to a square stripe pattern, as
shown in Figure 8(a). Therefore, they
simulate three-step phase shifting using the defocused square stripes, where
the phase shift is realized by shifting the
stripes spatially. This method turns out
to work quite well, and the phase error is
(a)
(b)
(c)
negligible, given a proper degree of projector defocusing and a proper width
FIGURE 8. Base patterns of binary phase shifting: (a) a square binary pattern, (b) a dithered binary patof the stripes. Using a DLP projector
tern, and (c) a density-modulated binary pattern.

IEEE SIgnal ProcESSIng MagazInE

May 2017

Table of Contents for the Digital Edition of Signal Processing - May 2017