IEEE Geoscience and Remote Sensing Magazine - December 2022 - 129

technology, large-scale distributed computing for remote
sensing big data has formed a relatively mature ecology,
which reduces the difficulty of building an HPC platform.
In addition, one of the advantages of cloud computing for
remote sensing big data processing is the distributed file
system, and more researchers are using cloud computing
technology to process and analyze remote sensing
big data. Hadoop and Spark are typical platforms that are
widely used. Scholars have done a great deal of research
on improving the computational efficiency of remote
sensing big data processing and analysis tasks based on
these two platforms.
Hadoop originated from the search application developed
by Google and was then developed into an open
source system by the Apache Foundation. The core of Hadoop
is the Hadoop distributed file system (HDFS) and
MapReduce (see Figure 1). The HDFS is used for mass data
storage, MapReduce is used for mass data calculation, and
the Hadoop parallel computing framework is suitable for
various scientific applications in remote sensing data processing.
Golpayegani and Halem [72] realized the parallel
programming of remote sensing data processing through
Hadoop MapReduce. The research results showed that the
code implementation of the Hadoop algorithm was simpler
than the basic algorithm. With increased data volume, it
Remote Sensing
Application
Satellite
Product
Earth Base
Stations
Hadoop Ecosystem
MapReduce
HDFS
Processing
Segmentation
Processing
Computation
Merging
Storage
Data
Management
Collection
Unit
Filtration
had better computational efficiency than the traditional
parallel method.
Rathore et al. [153] proposed a real-time method for
continuous feature extraction and detection based on
remote sensing satellite images; realized the complete
process of data segmentation, calculation, merging, and
management by the Hadoop ecosystem; and achieved
near-real-time computational efficiency. Zou et al. [238]
put forward an abstract data format for various types of
remote sensing data to realize a unified description of the
data. By discretizing multidimensional remote sensing
data for distributed storage and calculation, MapReduce
could solve the complexity of the remote sensing algorithm,
which was applied to global vegetation drought
monitoring. With increased computing nodes, the time
cost was nearly linearly reduced, and the throughput was
nearly linearly increased.
Although Hadoop has been widely used for the parallel
processing of large-scale remote sensing data since
its launch, it cannot solve the problems of intensive disk
I/O operation and frequent network communication, and
it has difficulty further meeting the needs of application
scenarios such as the real-time processing and analysis
of massive and heterogeneous remote sensing data [84].
Apache Spark is a fast general-purpose computing engine
Interpretation
Communication Infrastructure
FIGURE 1. The architecture of the remote sensing data real-time processing system based on the Hadoop ecosystem [153].
DECEMBER 2022 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
129
Preprocessing

IEEE Geoscience and Remote Sensing Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2022