IEEE Geoscience and Remote Sensing Magazine - March 2021 - 54

analysis (MRA) and component substitution (CS)] are
compared with methods belonging to the third generation of pansharpening, represented by variational optimization-based (VO) and machine learning (ML) techniques. The benchmark is tested on different scenarios
(from urban to rural areas) acquired by different commercial sensors [i.e., IKONOS (IK), GeoEye-1 (GE-1), and
WorldView-3 (WV-3)]. Both quantitative and qualitative
assessments and the computational burden are analyzed
in this article, and all of the implementations have been
collected in a MATLAB toolbox that is made available to
the community.
OVERVIEW OF PANSHARPENING
The main objective of satellite remote sensing is to
provide accurate reproductions of the Earth's surface.
This can be obtained by improving hardware to resolve
many more details in both space and frequency domains. However, it turns out to be a hard task because
of the stringent constraint of the signal-to-noise ratio
of satellite products. A way around the problem consists of combining multiple images with complementary features to get high-quality products through signal
processing [1].
Pansharpening, which stands for panchromatic sharp-
ening, is a powerful and well-known image fusion
methodology involving the fusion of an MS and a PAN
image to provide a final outcome with the same spatial resolution as the PAN image and the same spectral
resolution as the MS image. Pansharpening has proven
to be useful for both visual interpretation (see, e.g., its
use in commercial software such as Google Earth and
Bing Maps) and as a preliminary step for higher-level
processing (see, e.g., the use of pansharpened images
for crop mapping [2], land cover monitoring [3], and
anomaly detection [4]). Some pansharpening techniques have also been extended to slightly different
issues, i.e., the fusion of PAN and hyperspectral (HS)
images [5], [6] and the combination of MS and HS
data [7]-[10].
BACKGROUND AND RELATED WORKS
The central role played by pansharpening methods within
remotely sensed image fusion techniques is testified to by
a comprehensive book on the subject [11] as well as review
articles as [12]. Actually, specifically devoted surveys have
populate the literature in recent years [13]-[17], corroborating the growing interest in this topic.
Several taxonomies have been applied to pansharpening algorithms. For each case, relationships among the
different classes can be found. In this article, we adopt
a broad classification that is pointed to practical use. In
par ticular, we defined four main classes characterized by well-distinguished properties that are crucial in
the selection of the most appropriate algorithm for the
specif ic application.
54

The first two classes (CS and MRA) make up the group
of classical approaches, which was deeply analyzed in a
previous study [14]. However, as will be widely illustrated
in the following, significant improvements have been obtained in recent years, motivating the introduction of a
large number of novel methods in this article. Indeed, since
the recent developments preserve the key characteristic of
high computational efficiency, classical algorithms are still
of fundamental importance.
Instead, the two remaining classes (VO and ML) represent the main emerging lines of research in the last few
years [18], [19]. Their unequivocal objective is to achieve
the uncompromised quality of the fused product, which often induces a higher level of complexity and a consequent
increase of computational effort.
We start with a brief description of the four classes and
go on to describe the specific methods selected for implementation and assessment. Finally, we review the literature
related to the debated problem of the quantitative assessment of fused products.
COMPONENT SUBSTITUTION
The CS approaches are also referred to as spectral methods.
They are based on the projection of the original MS image in
a transformed domain. The objective is to ease the replacement, either partial or total, of the spatial information with
that of the PAN image. Thanks to their simple implementation, many pioneering pansharpening algorithms belong
to the CS class. For example, those exploiting intensity-huesaturation (IHS) [20], [21] and principal component analysis (PCA) [22], [23] date back to the early 1990s. However,
the appealing visual features of the final products and the
robustness against spatial misalignments have favored the
development of several other algorithms belonging to this
class. Other methods, for example, the Gram-Schmidt (GS)
expansion [24], the matting model [25], nonlinear IHS [26],
and nonlinear PCA [27], have been exploited for deriving
more specific image representations with an easily identifiable spatial component.
Considering the different image transforms, a plethora
of combination algorithms have been proposed for introducing PAN spatial information into the original MS data.
The main contribution of the recent literature has been the
improvement of the injection rules to the extent that the
new methods can be considered second-generation techniques. The focus of these studies has centered on accurately modeling the relationship between the pixel values
of the PAN image and those of the MS channels. An iterative fitting algorithm was proposed in [28], but the most
used methods rely on statistical modeling. This methodology was implemented, in different forms, by the partial
replacement adaptive CS (PRACS) [29] and the adaptive
GS (GSA) [30]. A more comprehensive study can be found
in [31].
Another fruitful line of research has originated in the
local application of CS algorithms. This principle has been
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

MARCH 2021
IEEE Geoscience and Remote Sensing Magazine - March 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - March 2021
