IEEE Geoscience and Remote Sensing Magazine - June 2021 - 73

prior probabilities for all of the mixture classes, including
the mean vectors and covariance matrices of the endmember
classes. The learned prior probabilities are passed to the
cost function to help with the reconstruction of the final
HrHS image X.
Yokoya et al. regarded (24) as the coupled NMF (CNMF)
problem [128] and introduced multiplicative-update rules
to optimize (24). First, CNMF utilizes YEAR
cost function to update E and Ah
-
trix E initialized by VCA. Here, AARh
2
F
matrix from HS images. Second, by initializing EG ,E
ZG .EA
2
-
which is the endmember matrix from the MS image, CNMF
again utilizes the multiplicative-update rules to update E
from the cost function
as the
with the endmember ma=
is the abundance
m =
F Finally, the HrHS image
X is reconstructed from EA . The author of [133]-[135]
also utilize the CNMF framework to fuse HS-MS images.
Differently, the authors in [133] and [135] introduced a
nonnegative dictionary-learning strategy while the works
of [134] proposed the proximal-alternating linearized-minimization
algorithm to update E and A.
On the basis of (24), Wang et al. further regularized
X with nonlocal, low-rank Tucker decomposition [136].
The improved nonlocal, Tucker-decomposition-regularized
CNMF model [136] was solved by the multiblock ADMM
and achieved remarkable fusion results. It indicates that the
additional regularizers on X can further improve fusion
accuracy. From another side, it is necessary to make a tradeoff
between the complex models with higher accuracy and
computation efficiency for real, large-scale HS-MS imagefusion
tasks.
ORTHOGONAL-SUBSPACE-BASED METHODS
Another common assumption in HS-MS fusion is that the
spectral information of X underlies an orthogonal subspace
whose dimension is much smaller than that of the number of
bands B [101], [129], i.e., XEA= with E ,,A R
is an orthogonal matrix with EE .Ik
and kB .% E<
min
XE A
,,
s.t.
YXRZ GX
XEAE EI
-+ -
==R
,.
k
!!
<
RBk
fore, the subspace-based model is formulated as
,
22
FF
(25)
Although an additional spectral subspace prior is exploited,
the optimization of (25) still faces several challenges. First,
if
kb ,& that is, the dimension number of the subspace is
larger than that of the bands' number of the MS image,
the optimization of (25) is an underestimation problem.
Therefore, to ensure a reasonable solution, prior information
of coefficient A needs to be exploited. The authors in
[129] pretrain a dictionary to represent A and update A
via sparse representation. The HS superresolution (HySure)
in [130] assumes that A appears in the spatial smoothness
structure and regularizes A with band-by-band TV. In
[137], the optimization of A translates to a Sylvester equation
and a fast fusion method (FUSE) for (25) is proposed.
JUNE 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
=
##
kMN
ThereSecond,
the optimization of orthogonal matrix ER
is another
challenge due to the nonconvex nature of (25). One
new approach [129], [130], [137] is to pre-estimate E from Y
in advance and fix variable E during the optimization of (25).
Specially, FUSE [137] adopted PCA while HySure utilized
VCA to extract E from .Y Another strategy is to regularize the
update of E and A as a coupled matrix factorization problem,
using a blind dictionary-learning strategy to update E [138]. A
hybrid, inexact block-coordinate descent [139] is introduced
to exactly estimate .E
TENSOR-BASED METHODS
The aforementioned subspace-based methods utilize lowrank
matrix decomposition to exploit the low-rank property
of the reshaped high-spatial-resolution HS image .X However,
the original HS image is a 3D tensor, and therein the
researchers introduce tensor decomposition to simultaneously
capture the spatial- and spectral-low-rank property.
The coupled sparse tensor factorization (CSTF) approach
[140] utilized Tucker decomposition, presented as
XO EE E ,
i = , O #C,
s.t. EE I
i
= ## #
R
11 2233
(26)
to regularize the high-spatial-resolution HS image X .
In (26), the core tensor O is assumed to obey the sparse
property, and Ei
is the orthogonal matrix of the ith dimension.
Subsequently, CP decomposition [141], tensor
train decomposition [142], tensor ring decomposition
[131], [143], and so on are utilized to regularize .X Furthermore,
nonlocal LRTD is also investigated for the fusion
task [144]-[146].
It is worth noting that the unmixing-, orthogonalsubspace-,
and tensor-based methods share the common
belief that the spectral space of X should lie in the lowdimensional
space. Unmixing-based approaches interpret
the low-rank property as endmembers and abundances,
which are assumed to be nonnegative, while orthogonalsubspace-
and tensor-based methods ignore the nonnegative
restrict. Unmixing-based approaches are interpretable
from the physical meaning but suffer from unstable convergence
during the optimization. Orthogonal-subspace- and
tensor-based methods lose physical meaning but can be
optimized more elegantly.
Very recently, some preliminary works that perform the
fusion task by means of DL-based methods [147]-[152]
have shown effective and competitive fusion performance.
A similar problem that existed in these methods is model
interpretability and rationality. Clearly explaining the intrinsic
meaning in each layer of deep networks would contribute
to better modeling of the fusion task and further
obtaining higher-quality products.
EXPERIMENTAL STUDY
In this section, we select the unmixing-based methods
CNMF [128], [191] and 2015 International Conference on
73
IEEE Geoscience and Remote Sensing Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2021
