IEEE Geoscience and Remote Sensing Magazine - December 2020 - 73

(KSGDA) and kernelized GCDA (KCGDA), the main difference lies in the computation of the adjacency matrix, which
can be performed in the kernel space by solving the general
kernel coding problem as follows:
	

min X(W) s.t.
W

U(X m) W - U(X m)

2
F

# e, (25)

LSQMI DERIVATIVE
Due to the difficulty in accurately computing the derivative of the QMI estimator, LSQMI was further extended to
a computationally effective LSQMI derivative by estimating the derivative of QMI instead of QMI itself [84]. In that
article, Tangkaratt et al. [84] demonstrated a more accurate
and efficient derivative computation of QMI.

where X(W) can be selected to be either the sparsity-prompting term W 1 of KSGDA or the dense (or collaborative)
term W 2F of KCGDA. In [78] and [74], the solutions in
(25) were theoretically guaranteed in the same way by solving (22) and (23) using the alternating-direction method of
multiplier (ADMM) [79] and least-square regression with
Tikhonov regularization [80], respectively.

JOINT AND PROGRESSIVE LEARNING STRATEGY
Another MI-free estimation group is latent SL (LSL). One
representative LSL performs FE and classification simultaneously in a joint-learning (JL) fashion [85]. With an expected output HX m, the process can be modeled as

RIRL
RIRL provides a new insight from the regression point of
view to model the FE behavior by bridging the training samples with the corresponding labels rather than indirectly using the label information in the form of a graph or affinity
matrix in DAFE-based methods.

where Yl ! R k # m and H ! R d # p are defined as the one-hot
encoded label matrices and the latent subspace projections,
respectively. P k ! R k # d denotes the regression matrix that
connects the learned subspace and the label information.
Yl can be formulated as

LEAST-SQUARES DIMENSION REDUCTION
We begin with sliced inverse regression [81], which is a
landmark in SFE techniques. It assumes that the pairwise
data {(x i, y i)} mi = 1 are conditionally independent on the
to-be-estimated subspace features {z i}mi = 1, formulated as
(X = Y )| Z. Following this rule, the least-squares dimension reduction (LSDR) proposed by Suzuki and Sugiyama
[82] attempts to find a maximizer of the squared-loss MI
(SMI) to satisfy the previously mentioned independence
assumption. The projections P for LSDR can be searched
by optimizing the following maximization problem:

min Yl - P k HX m
P k, H

R1
S
S0
S
Yl = S
S0
S
SS
0
T

	

P

And the SMI to measure a statistical dependence between
two discrete variables is defined as

/ / p(z) p(y)d p(pz(z) ,py(y) ) - 1 n , (27)
2

SMI(Z, Y ) =

z!Z y!Y

where p(:) is the probability distribution function.
LEAST-SQUARES QUADRATIC MI
Limited by the sensitivity of MI to outliers, Sainui and Sugiyama [83] designed a more robust least-squares quadratic
MI (LSQMI) on the basis of a QMI criterion; hence, let us
define the QMI as

2
F

a
+ 2 Pk

0 f 0
1 f 0
f f
0 f 1
f f
0 f 0

2
F

f
f
f
f
f
f

s.t. HH T = I, (29)

0VW
0W
W
W
0W
W
W
1W
X

1
2
f
. (30)
j
f
k

In [85], the model's solution was proven to be a closed
form. Moreover, in [86], Hong et al. explored an LDA-like
graph as a regularizer to learn a spectrally discriminative
feature representation; thus, (29) becomes
	

min Yl - Pk X m
P k, H

2
F

a
+ 2 Pk

max SMI(Z, Y ) s.t. PP T = I. (26)

	

	

	

2
F

b
+ 2 tr (HX m LX mT H T)

s.t. HH T = I.

(31)

Beyond the JL-based models, Hong et al. [87] established a novel multilayered regression framework by following a joint and progressive learning strategy (JPlay).
With the layerwise autoreconstruction mechanism effective against the spectral variabilities caused by complex noises and atmospheric effects, the linearized JPlay
breaks through the performance bottleneck of traditional linear methods. More specifically, we have the resulting model
minq Yl - P k H q f H 1 X m

P k,{H l} l = 1

2
F

a
+ 2 Pk

2
F

q

	

b
+ 2 / tr (H l X l - 1 LX lT- 1 H lT )
l=1



q

	

QMI(Z, Y ) =

/ / (p(z, y) - p(z) p(y))2. (28)

z!Z y!Y

Similarly, we solve (26)-like optimization problem by replacing SMI with QMI.
DECEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

c
2
+ 2 / X l - 1 - H lT H l X l - 1 F,
l=1
s.t. X l = H l X l - 1, X l $ 0, x i

2

# 1,

(32)

where the soft constraint x i 2 # 1 can be used to relax
the orthogonality. It is worth noting that such a JL-based
73
IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020
