IEEE Robotics & Automation Magazine - December 2015 - 123

earlier, in the PCL we refer to these image-like 3-D representations as organized point clouds.
It is assumed that consecutive clouds acquired from the
sensor are temporally (and, thus, spatially) close to each
other. For this to hold, not only must the device have a sufficiently high frame rate, but the pipeline must also process
frames at a comparable speed. This is the working assumption of cloud registration algorithms deployed within
SLAM and Kinect Fusion-like frameworks [29]. Under
these conditions the point clouds are close enough to allow
ICP to reach a good local minimum in most cases, without
the need for the initial alignment procedure, and with just
a few iterations. We will briefly walk through the steps of
the pipeline.
Preprocessing-Filtering and Fast Feature Estimation
This type of camera exhibits various forms of noise. Hence,
initial filtering becomes necessary when the cameras are used
to acquire accurate 3-D models. There are multiple ways in
which noise can be efficiently eliminated from generic point
clouds, but none of them consider the difficult quantization
effects seen in this new type of data. An edge-preserving filtering method, the bilateral filter, has been adapted from the
field of 2-D image processing (" FastBilateralFilter). It smoothes more when neighboring pixels are similar
(flat regions), and less when there are jumps, therefore, it does
not affect the edges. Listing 13 provides a usage example.
Accelerating the computation of normals can also be
done by using the grid structure of the RGB-D data. The rasterized depth images can be used to speed up the normal estimation process using integral images [30]. The advantage
of this method is that it requires only a linear preprocessing
stage while allowing for computing mean vectors and normals within a rectangular area of the image in constant time
(" IntegralImageNormalEstimation).
Registration Pipeline
For aligning a sequence of RGB-D images, we present a hybrid
registration pipeline. Because of the high data rate of these sensors (640 # 480 = 307, 200 points at 30 Hz), the measurements need to be sampled so that the registered point clouds
contain only 10% of the original data. As described in the "Selection-Sampling Representative Subsets" section, there are
many approaches available for sampling the point cloud.
Normal Space Subsampling
A particularly robust way to register RGB-D point clouds
is to perform normal space sampling (" NormalSpaceSampling). This method consists of binning each point
into M 3 bins based on the normal angle around each of
the x, y, and z-axis, respectively. Next, a fixed number of
samples is extracted randomly from each bin. This ensures
that the resulting point cloud has samples at locations from
all the differently oriented surfaces in the scene, increasing
the probability that the registration will converge to the
global minimum.

Correspondence Estimation and Filtering
Having only a fraction of the points left, using a complex
search data structure is not that prohibitively slow anymore.
Consequently, we apply the correspondence estimation
based on kd-trees (discussed in the "Matching-Closest Points
Correspondence Estimation" section) rather than the projection-based estimation.
To reject misleading correspondences, we employ a combination of two correspondence filters (discussed in the "Rejecting and Filtering Correspondences" section): a filter
based on normal compatibility (" CorrespondenceRejectorSurfaceNormal) to reject point pairs with
mismatched normals, and a distance filter using the median
distance between correspondence pairs (" CorrespondenceRejectorMedianDistance) to eliminate the
pairs with outlying distances.
Transformation Estimation and Weighting
Using the resulting small set of robust correspondences, the
transformation between the two point clouds is computed
using the weighted point-to-plane distance metric
(" TransformationEstimationPointToPlaneWeighted). Because our normals have been computed on
the original point clouds, they approximate the underlying
surface well. As a result, the point-to-plane error metric is
preferred as a better estimation of the distance between the
source and target surfaces, as opposed to point-to-point measurements. The weighting of the correspondences can be
done using a sensor noise model. One such example was proposed by [13] and considers the standard deviation of the
depth to increase quadratically with the distance of the
scanned surfaces from the image plane. As such, the weight
for the correspondence between points p and q is taken as
the maximum of the standard deviation of the two points
under the noise model
w ^ p, qh = max ^v z ^ p z h, vz ^q z hh,

with v z ^ z h = 0.0012 + 0.0019 ) ^z - 0.4h .
2

(5)
(6)

As convergence criteria for the ICP algorithm (discussed
in the "Termination Criteria" section), a good configuration
is to use the number of iterations (chosen based on the time
we have allocated for registration) and the length of the incremental translation and rotation from the last iteration
(chosen to be lower than the expected camera movement between frames).
Typical Registration Results
For the purpose of the experiment, we recorded sequences of
RGB-D frames at 30 Hz. Each frame is registered in a pairwise fashion against the previous one. Results are shown in
Figures 1 and 10. Note that if a large number of frames is registered pairwise, individual alignment errors accumulate.
There are numerous techniques in the literature to solve this
issue, such as loop closing via graph optimization or forms of
global registration.
DECEMBER 2015

IEEE ROBOTICS & AUTOMATION MAGAZINE

123

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2015