IEEE Geoscience and Remote Sensing Magazine - December 2020 - 43

To overcome the drawbacks of early data sets, new benchmark data have been made available in recent years. Currently,
mainstream PCSS benchmark data sets are from either lidar or
RGB-D systems. A nonexhaustive list of these data sets follows.
SEMANTIC3D.NET
Semantic3D.net [34] is a representative, large-scale outdoor
TLS PCSS data set. It constitutes a collection of urban scenes
with more than four billion labeled 3D points for PCSS purposes only. Those scenes contain a range of urban objects, divided into eight classes, including manmade terrain, natural
terrain, high vegetation, low vegetation, buildings, hardscape,
scanning artifacts, and cars. In consideration of the efficiency
of different algorithms, two types of subdata sets were designed, Semantic-8 and Reduced-8. Semantic-8 is the full data
set, while Reduced-8 uses training data in the same way as Semantic-8 but includes only four small subsets as test data. This
data set can be downloaded at http://www.semantic3d.net/.
To learn about the performance of different algorithms on this
data set, refer to [2], [67], and [112].
STANFORD LARGE-SCALE 3D INDOOR SPACES DATA SET
Unlike Semantic3D.net, the Stanford Large-Scale 3D Indoor
Spaces Data Set (S3DIS) [44] is a large-scale indoor RGB-D
data set, which is also a part of the 2D-3D-S data set [137]. It is
a collection of more than 215 million points, covering an area
of more than 6,000 m2 in six indoor spaces originating from
three buildings. The main covered areas are for educational
and office use. Annotations in S3DIS have been prepared at
an instance level. Objects are sorted as structural or movable
elements, which are further divided into 13 classes (structural
elements: ceiling, floor, wall, beam, column, window, door;
movable elements: table, chair, sofa, bookcase, board, clutter
for all other elements). The data set can be requested from
http://buildingparser .stanford.edu/dataset.html. To learn the
performance of different algorithms on this data set, see [2],
[70], [100], and [119].
VAIHINGEN POINT CLOUD SEMANTIC LABELING DATA SET
The Vaihingen Point Cloud Semantic Labeling Data Set [31]
is the most well-known published benchmark data set in the
photogrammetry and remote sensing field in recent years.
A collection of ALS point clouds, it consists of 10 strips captured by a Leica ALS50 system with a 45° field of view and
500-m mean flying height over Vaihingen, Germany. The
average overlap between two neighboring strips is around
30%, and the median point density is 6.7 pts/m2 [31]. This
data set had no label at a point level at first. Niemeyer et al.
[87] first used it for a PCSS test and labeled points in three
areas. Now the labeled point cloud is divided into nine classes
as an algorithm evaluation standard. Although this data set
has significantly fewer points compared with the Semantic3D.
net and S3DIS data sets, it is an influential ALS data set for
photogrammetry and remote sensing. The data set can be
request--ed from http://www2.isprs.org/commissions/comm3/
wg4/3d-semantic-labeling.html.
DECEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

PARIS-LILLE-3D
The Paris-Lille-3D data set [36], published in 2018, is a new
benchmark for PCSS. It is an MLS point cloud data set with
more than 140 million labeled points, including 50 different
urban object classes along 2 km of streets in two French cities,
Paris and Lille. As an MLS data set, it also could be used for
autonomous vehicles. As this is a recent data set, only a few
validated results are shown on the related website. This data set
is available at http://npm3d.fr/paris-lille-3d.
SCANNET
ScanNet [43] is an instance-level indoor RGB-D data set that
includes both 2D and 3D data. In contrast to the benchmarks
already mentioned, ScanNet is a collection of labeled voxels
rather than points or objects. ScanNet v2, the newest version
of ScanNet, has collected 1,513 annotated scans with approximately 90% surface coverage. In the semantic segmentation
task, this data set is marked in 20 classes of annotated 3D voxelized objects. Each class corresponds to one category of furniture. This data set can be requested from http://www.scan-net
.org/. To learn about the performance of different algorithms
on this data set, refer to [70], [120], [123], and [124].
POINT CLOUD SEGMENTATION TECHNIQUES
PCS algorithms are based mainly on strict handcrafted features
from geometric constraints and statistical rules. The main process of PCS aims at grouping raw 3D points into nonoverlapping regions. Those regions correspond to specific structures or
objects in each scene. Since no supervised prior knowledge is
required in such a segmentation procedure, the delivered results have no strong semantic information. Those approaches
could be categorized into four major groups: edge based, region growing, model fitting, and clustering based.
EDGE BASED
Edge-based PCS approaches were directly transferred from 2D
images to 3D point clouds, which were mainly used in the very
early stage of PCS. As the shapes of objects are described by
edges, PCS can be solved by finding the points that are close
to the edge regions. The principle of edge-based methods is
to locate the points that have a rapid change in intensity [16],
which is similar to some 2D image segmentation approaches.
According to the definition from [138], an edge-based segmentation algorithm is formed in two main stages: 1) edge
detection, where the boundaries of different regions are extracted; and 2) grouping points, where the final segments are
generated by grouping points inside the boundaries extracted
by edge detection. For example, in [139], the authors designed
a gradient-based algorithm for edge detection, fitting 3D lines
to a set of points and detecting changes in the direction of unit
normal vectors on the surface. In [140], the authors proposed
a fast segmentation approach based on high-level segmentation primitives (curve segments), in which the amount of data
could be significantly reduced. Compared to the method presented in [139], this algorithm is both accurate and efficient,
but it is only suitable for range images and may not work for
43

http://www.SEMANTIC3D.NET http://www.Semantic3D.net http://www.npm3d.fr/paris-lille-3d http://www.semantic3d.net/ http://www.scan-net.org/ http://www.scan-net.org/ http://www.Semantic3D.net http://buildingparser.stanford.edu/dataset.html https://www2.isprs.org/commissions/comm2/wg4/benchmark/3d-semantic-labeling/ https://www2.isprs.org/commissions/comm2/wg4/benchmark/3d-semantic-labeling/

IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020