IEEE Geoscience and Remote Sensing Magazine - June 2021 - 76

steps are jointly optimized in an SSL model, which tends
to yield more discriminative-feature representations.
More specifically, we briefly review and detail some representative
approaches belonging to the aforementioned
two groups in the following sections.
THE CORE IDEA OF CML IS
TO FIND A NEW DATA SPACE
WHERE THE INFORMATION
CAN BE EXCHANGED
EFFECTIVELY ACROSS
DIFFERENT MODALITIES.
MA-BASED APPROACH
MA is capable of aligning multiple
modalities on manifolds
into a latent subspace, achieving
a highly effective knowledge
transfer [160]. Due to its
interactive learning ability,
MA is a good fit for large-scale
RS image classification. In
[161], domain adaptation was
investigated to reduce the gap
between the source and target domains of HS data for land
cover classification. By simultaneously considering labeled
and unlabeled samples, Tuia et al. [162] used SSMA techniques
[163] to align multiview RS images onto the manifold
space using attempts to eliminate the effects of image
variants caused by different views. Matasci et al. [164] modified
the classic transfer component analysis [165], making
it applicable to the land cover classification of RS images.
Moreover, the kernelized MA approach presented in [166]
projected the multimodal RS data to a higher-dimensional
space and aligned them in a nonlinear way. Hu et al. [167]
deeply reviewed SSMA methods with respect to the fusion
classification of HS and polarimetric SAR images. Based on
the work in [167], the same investigators made full use of topological
data analysis and designed a new graph structure
for optical (e.g., HS) and SAR data fusion [168].
Mathematically, the MA idea can be implemented by
solving the following nonconvex model:
,
" ,mmin B
U s=1
where A, B, and C are
A
B
C
===2
1
2
1
2
1
/
/ / /
/ / / /
/ / /
m
n
n
n
n
p=1 q=1 i =1 j =1
m
p=1 q=1 i =1 j =1
m
Uy Uy Wij
,
t
t=1 i =1 j =1
By minimizing the problem (28), " ,m
=
U s =1 can be estimated
via generalized eigenvalue decomposition. We then have
XU .Yss
Three different graphs need to be precomputed
in (28), including the similarity graph, i.e., W ,sim
R
W =sim
T
S
S
S
S
S
76
W
W
W
1,1
2,1
sim
sim
W
W
W
1,2
2,2
hh
,1
sim
sim
mm,2
sim
sim
g
g
j
g
W
W
1,
2,
h
sim
m
m
V
sim
sim
W ,mm
X
W
W
W
W
W
;
(29)
PU ,mmin
, "
2
1
s s=1
s.t.
MPUY PU
UU I
s
-+WX+
R==
s
s
2
F
s sm
,, ,,
1 f
() ^h "
s s,m
=1
(32)
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE JUNE 2021
t
i
t
t
j
2
2
t
Uy Uy W
Uy Uy W
p p
i
p p
i
q q
j
q q
j
2
2
2
2
.
m
m
n
n
sim
ij
,
,
dis
ij
,
,
AC
+
(28)
the dissimilarity matrix, i.e., W ,dis
R
W =dis
T
S
S
S
S
S
W
W
W
1,1
2,1
dis
dis
W
W
W
1,2
2,2
dis
dis
hh
,1
mm,2
dis
dis
g
g
j
g
W ,t
W
W
1,
2,
h
dis
dis
dis
m
m
V
W ,mm
X
W
W
W
W
W
;
(30)
and the topology structure for each single modality obtained
by the knn graph, i.e.,
R
Wt =
T
S
S
S
SS
ly, are given by
W
ij,
sim
W
ij,
dis
W ,ij
t
=
=
=
k *
(
(
1,
0,
1,
0,
if and
otherwise,
yy k
p
i
if and
otherwise,
yy k
p
i
*exp
yy
-
ij
2v2
2
2
,
q
j
"
!
C
C
ifyy
p
i
k
0, otherwise,
where z ^h denotes the KNNs of * .
SHARED SUBSPACE LEARNING-BASED APPROACH
Due to the lack of direct relational modeling between the
learned features and label information, MA-based approaches
fail to activate connections across modalities effectively
[159], thereby yielding a relatively weak transferability
between different modalities, particularly heterogeneous
data. There have been some tentative works in recent years,
providing potential solutions to overcome these challenges.
For example, Hong et al. [169] for the first time proposed
a supervised common subspace learning (CoSpace) model
to learn a latent, discriminative subspace from HS-MS correspondences
for the CML-related classification problem.
Beyond it, the same authors [170] fully tapped the potential
of the CoSpace by learning the data-driven graph structure
from both labeled and unlabeled samples, yielding a learnable
MA (LeMA) approach. Moreover, the authors in [171]
thoroughly investigated and analyzed different regression
techniques, i.e.,
, norm ridge regression and -1
2 -
, norm
sparse regression, in CoSpace. In [172], a semisupervised
graph-induced aligned learning (GiAL) was developed by
jointly regressing labels and pseudolabels.
Accordingly, these methods can be generalized to be
a unified model [169] to address the CML's problem in a
regression-based fashion:
!z ();
q
j
q
j
W ,11
t
W ,22
d
hh
In equations (29)-(31), ,, and W ,ij,
WW,
ij
ij,
sim dis
t
g
g
j
g
h
t
V
W ,mm
X
W
W
W
WW
.
(31)
respective
IEEE Geoscience and Remote Sensing Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2021
