IEEE Geoscience and Remote Sensing Magazine - June 2021 - 72

particular, significant effectors have recently been made to
enhance the spatial or spectral resolutions of HS images
by means of ML techniques. Figure 11 displays the fusion
process of HS-MS images to generate the HrHS image.
Suppose that we have the low-spatial resolution HS image
Y
Rmn B## and the high-spatial-resolution MS image
Z RMN b## with Mm ,Nn
!
!
X RMN B
##
!
Z are formulated as
YXRNH
ZGXNM
=+
=+ ,
(21)
(22)
HS PANSHARPENING IS A
HEURISTIC WAY TO
PERFORM HS-MS FUSION,
WHICH HAS BEEN WIDELY
APPLIED IN HS IMAGEENHANCEMENT
TASKS.
where XY Z,, are the reshaped
matrices along the spectral
dimension of
X ,, ,YZ respectively,
R is the mixed-cyclic
convolution and downsampling
operator, G is the
spectral-response function of
the MS image sensor, and NH
and NM
are the corresponding
MS-HS noise. To unify
different observation models
[126]-[131], NH
and NM
are assumed to be the independent,
identically distributed Gaussian noise. Via the maximum
a posteriori (MAP) estimation method and Bayes
rule [127], [129], [130], the following nonconvex optimization
model is obtained:
mi YXRZ GX ,
Xn -+ -
22
FF
(23)
where R and G are assumed to be known (in [126] and [129],
R and G are estimated in advance of the optimization, and,
&& and Bb :& the fusion purpose
is to generate the high-spatial-resolution HS image
.
The degradation models from X to Y and
as mentioned in [129] and [130], the optimization of (23) is
an NP-hard problem and an overestimation of Z will result
in unstable fusion results. Therein, the additional property
of X and prior regularizers should be exploited in the optimization
model (23). It should be noted, however, that the
two functions R and G can be given according to the known
sensors and can also be learned or automatically estimated
from the data themselves.
HS pansharpening is a heuristic way to perform HS-MS
fusion [132], which has been widely applied in HS imageenhancement
tasks. Component substitution and multiresolution
analysis are the two main types of pansharpening
techniques. The former one aims to inject detailed
information about MS images into the low-resolution HS
image, thereby generating the high-resolution HS product.
The latter " pansharpens " the HS image by linearly combining
MS bands to synthesize a high-resolution HS band
using regression techniques. Another group used for the
HS-MS fusion task is the subspace-based model, which
roughly consists of Bayesian and unmixing-based methods
(see [126]). Unlike pansharpening, the subspace-based
approaches project the to-be-fused MS and HS images to a
new space where the dimension is generally smaller than
that of the unknown high-resolution HS image by means
of the probability-driven Bayesian estimation (Bayesianbased
methods) or the SU-guided matrix joint factorization
(unmixing-based methods).
In the following sections, we focus on the subspace
methods and review the related HS-MS image-fusion methods
from a nonconvex modeling perspective. A more detailed
review can be found in [126] and [132].
HS Image
UNMIXING-BASED METHODS
HU [5], [101] assumes that the mixed class of an HS image
can be decomposed to the collection of constituent spectra
(endmembers) and their corresponding proportions (abundances).
With an LMM assumption, the different endmembers
do not interfere with each other [5]. By embedding the
LMM model into (23), we can obtain the following general
unmixing-based approaches:
min YXRZ GX-+ -
XE A,,
R
22
FF
,
s.t. XEAEAA MN==<
Fusion
,, ,,$01 1
(24)
HrHS Image
where EA, are the endmember and abundance matrices,
which are assumed to obey the nonnegative and abundant
sum-to-one constraints. Generally, nonlinear unmixing
models [5] can also be utilized for the fusion
task of HS-MS images. However, due to the generality of
LMM models, we focus on the review of LMM-based fusion
approaches.
Eismann et al. proposed an MAP estimation method
MS Image
FIGURE 11. An MS-HS fusion image used to generate the HrHS image.
72
to deduce a cost-objective function and introduced a stochastic
mixing model (SMM) to embed LMM into the cost
function [127]. The MAP-SMM method tries to estimate the
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE JUNE 2021
IEEE Geoscience and Remote Sensing Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2021
