IEEE Geoscience and Remote Sensing Magazine - March 2019 - 10

include land cover classification, urban-rural definition,
target identification, and geological mapping (e.g., [30]). A
large area of current attention is the specific problem that
arises from the tradeoff in remote sensing between spatial
resolution and temporal frequency-in particular, the fusion of coarse-spatial/fine-temporal-resolution space-time
data sets with fine-spatial/coarse-temporal-resolution ones,
so as to provide frequent data with fine spatial resolution
[31]-[34]. This will be detailed in the "Pansharpening and
Resolution Enhancement" and "Multitemporal Data Fusion" sections. Land cover classification-one of the most
vibrant fields of remote sensing research [35], [36], attempting to differentiate between several land cover classes available in a scene-can substantially benefit from data fusion.
Another example is the tradeoff between spatial resolution and spectral resolution [Figure 1(b)] to produce fine
spectral-spatial resolution images, which play an important
role in land cover classification and geological mapping. As
can be seen in Figure 1(b), both fine spectral and spatial resolutions are required to provide detailed spectral information
and at the same time avoid the mixed-pixel phenomenon.
Further information about this topic can be found in the next
section. Elevation information provided by lidar and TLS [see
Figure 1(c)] can be used in addition to optical data to further
increase classification and mapping accuracy, in particular
for classes of objects that are
made up of the same materials (e.g., grassland, shrubs, and
FOr rEmOTE sEnsinGtrees). Therefore, the sections
bAsED GLObAL
"Point Cloud Data Fusion"
mOniTOrinG, ThErE
and "Hyperspectral and Lidar"
ALwAys ExisTs A
are dedicated to the topic of
elevation data fusion and their
TrADEOFF bETwEEn
integration with passive data.
spATiAL rEsOLUTiOn AnD
Furthermore, new streams of
TEmpOrAL rEvisiT
ancillary data obtained from
FrEqUEncy.
social media, crowdsourcing,
and scraping the Internet can
be used as additional sources
of information, together with airborne and spaceborne
data, for smart-city and smart-environment applications
and for hazard monitoring and identification. This young yet
active field of research is the focus of the "Big Data and Social
Media" section.
Many applications can benefit from fused fine-resolution
time-series data sets, particularly those that involve seasonal or rapid changes, as discussed in the "Multitemporal Data Fusion" section. Figure 1(d) shows the dynamic
of the changes for an area in Dubai, United Arab Emirates,
from 2001 to 2006 using time series of RGB and urban images. For example, keeping track of vegetation phenology
(the seasonal growing pattern of plants) is crucial to deforestation monitoring [37] and crop yield forecasting, which
militate against global food insecurity, natural hazards
(e.g., earthquakes and landslides), and illegal pollution activities (e.g., oil spills and chemical leaks). However, such
10

information is provided globally only at a very coarse
resolution, meaning that local smallholder farmers cannot benefit from the knowledge. Data fusion can be used
to provide the frequent data needed for phenology monitoring-and at a fine spatial resolution that is relevant to local
farmers [38].
Similar arguments can be applied to deforestation, where
frequent, fine-resolution data may aid in speeding up the timing of government interventions [37], [39]. The case for fused
data is arguably even greater for rapid-change events, for example, forest fires and floods. In these circumstances, the need for
frequent updates at a fine resolution is obvious. While these
application domains provide compelling arguments for data
fusion, there exist many challenges, including 1) the data volumes produced at a coarse resolution via such sensors as the
Moderate Resolution Imaging Spectroradiometer (MODIS)
and MERIS are already vast, meaning that data-set fusion
most likely needs to be undertaken on a case-by-case basis as
an on-demand service, and 2) rapid-change events require
ultrafast processing, meaning that speed may outweigh accuracy in such cases [40].
In summary, data fusion approaches in remote sensing
vary greatly, depending on the many considerations described previously, including the sources of the data sets
to be fused. In the following sections, we review data fusion approaches in remote sensing in terms only of the data
sources to be fused; however, the further considerations introduced previously are relevant to each section.
pAnshArpEninG AnD rEsOLUTiOn EnhAncEmEnT
Optical Earth observation satellites have tradeoffs in spatial, spectral, and temporal resolutions. Enormous efforts
have been made to develop data fusion techniques for reconstructing synthetic data that have the advantages of
different sensors. Depending on which pair of resolutions
has a tradeoff, these technologies can be divided into two
categories: 1) spatiospectral fusion to merge fine spatial and
fine spectral resolutions [see Figure 2(a)] and 2) spatiotemporal fusion to blend fine spatial and fine temporal resolutions [see Figure 2(b)]. This section provides overviews of
these technologies, including recent advances.
SPATIOSPECTRAL FUSION
Satellite sensors like WorldView and Landsat ETM+ can
observe the Earth's surface at different spatial resolutions
in different wavelengths. For example, the spatial resolution of the eight-band WorldView multispectral image is
2 m, but the single-band panchromatic (PAN) image has a
spatial resolution of 0.5 m. Spatiospectral fusion is a technique to fuse fine spatial resolution images (e.g., a 0.5-m
WorldView PAN image) with coarse spatial resolution images (e.g., a 2-m WorldView multispectral image) to create
fine spatial resolution images for all bands.
Spatiospectral fusion is also termed pansharpening when
the available fine spatial resolution image is a single PAN
image. When multiple fine spatial resolution bands are
ieee Geoscience and remote sensing magazine

march 2019

IEEE Geoscience and Remote Sensing Magazine - March 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - March 2019