IEEE Geoscience and Remote Sensing Magazine - December 2020 - 41

(fewer than 20 pts/m2), 2) dense (hundreds of pts/m2), and
3) multisource.
1)	 In the early stage, photogrammetric point clouds were
sparse, limited by matching techniques and computation
ability. At that time, only a few types of laser scanning systems were available, and they were not widely used. ALS
point clouds, which were the mainstream laser data, were
also sparse. Limited by point density, point clouds at this
stage were not able to represent land surface at an object
level. There was no specific demand for precise PCS or
PCSS. Researchers mainly focused on 3D mapping (DEM
generation) and simple object extraction (e.g., rooftops).
2)	 Computer vision algorithms, such as dense matching, and
high-efficiency point cloud generators, such as various lidar systems and RGB-D sensors, opened the big data era
of the dense point cloud. Dense and large-volume point
clouds created more possibilities in 3D applications while
also stimulating demands for practicable algorithms. PCS
and PCSS were proposed and became increasingly necessary, since only a class-level or instance-level point cloud
can further connect the virtual world to the real one. Both
computer vision and remote sensing need PCS and PCSS
solutions to develop class-level interactive applications.
3)	 From the perspective of general computer vision, research
on the point cloud and its related algorithms remains at
stage 2). However, driven by rapidly growing data from
spaceborne platforms and multisensors, remote sensing
researchers have a different understanding of point clouds.
New-generation point clouds, such as satellite photogram-

metric point clouds and TomoSAR point clouds, have stimulated demand for relevant algorithms. Multisource data
fusion has become a trend in remote sensing [62]-[64],
but current algorithms in computer vision are insufficient
for such remote sensing data sets. To fully exploit multisource point cloud data, more research is needed.
Table 1 provides an overview of basic information about
various point clouds, including point density, advantages, disadvantages, and applications.
POINT CLOUD APPLICATION
In studies about PCS and PCSS, requirements of specific applications drive the selection of data and algorithms. In this section, we outline most of the studies focusing on PCS and PCSS
reviewed in this article (Table 2). These studies are classified
according to their point cloud data types and working environments, such as urban, forest, industry, and indoor settings.
Several issues can be summarized from Table 2:
1)	 Lidar point clouds are the most commonly used data in
PCS applications. They have been widely used for buildings
(urban environments) and trees (forests). Buildings are also
the most popular research objects in traditional PCS applications. As buildings are usually constructed with regular
planes, plane segmentation is a fundamental topic in building segmentation.
2)	 Image-derived point clouds have been frequently used in
real-world scenarios. However, mainly due to the limitations of available annotated benchmarks, there are not many
PCS and PCSS studies on image-based data. -Currently, only

TABLE 1. AN OVERVIEW OF VARIOUS POINT CLOUDS.
POINT DENSITY
IMAGE
From sparse (<10 pts/m2)
DERIVED to very high (>400 pts/m2),
depending on the s- patial
resolution of the stereo- or
multiview images

ADVANTAGES

DISADVANTAGES

APPLICATIONS

With color (RGB, multispectral)
information; suitable for large
areas (airborne, spaceborne)

Influenced by light; accuracy depends on available precise camera models, image-matching
algorithms, stereo angles, image resolution, and
quality; not suitable for areas or objects without
texture, such as water or snow--covered regions;
influenced by shadows in images

Urban monitoring; vegetation monitoring; 3D
object reconstruction;
and more

LIDAR
 ALS

Sparse (<20 pts/m2); when
the survey distance is
shorter, the density is higher

High accuracy (<15 cm); suitable
for large areas; not affected by
weather

 MLS

Dense (>100 pts/m2);
when the survey distance is
shorter, the density is higher

High accuracy (centimeter level)

 TLS

Dense (>100 pts/m2);
when the survey distance is
shorter, the density is higher

High accuracy (millimeter level)

Small-area 3D reconstruction

 ULS

Dense (>100 pts/m2);
when the survey distance is
shorter, the density is higher

High accuracy (centimeter level)

Forestry surveying; mining surveying; disaster
monitoring; and others

RGB-D

Middle density

Cheap; flexible

Close range; limited accuracy

Indoor reconstruction; object tracking; human pose
recognition; and others

InSAR

Sparse (<20 pts/m2)

Global data are available; compared to ALS, complete building
façade information is available;
4D information; middle accuracy; not affected by weather

Expensive data; ghost scatterers; preprocessing techniques needed

Urban monitoring;
forest monitoring; and
others

DECEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

Urban monitoring; vegetation -monitoring; -power
line d
- etection; and more
Expensive; affected by mirror reflection; long
scanning time

HD map; urban monitoring

41
IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020
