IEEE Geoscience and Remote Sensing Magazine - December 2020 - 70

regression-induced representation learning (RIRL)-are
emphatically investigated and compared by clarifying their
similarities and differences as well as pros and cons, as
briefly illustrated in Figure 7.
DAFE
Generally speaking, DAFE seeks to find an optimal projection or transformation matrix P ! R p # d (d is the dimension of the subspace to be estimated) by optimizing certain class-relevant separation criteria associated with the
label information. In this process, the estimated subspace
Z ! R d # n, which consists of a series of vector z i, can be obtained by projecting the samples X m = " x i ,mi = 1 ! R p # m onto
a decision boundary, which can be generally expressed as
Z = P T X. Each vector z i in Z can be collected by P T x i . Depending on the different types of label embedding, DAFE
can be subdivided into LDA and its variants, graph-embedding-based discriminant analysis (GDA) and its extensions,
and kernelized discriminant analysis (KDA).
LDA AND ITS VARIANTS
Traditional LDA linearly transforms the original data into
a discriminative subspace by maximizing the Fisher's ratio
in the form of the generalized Rayleigh quotient, that is,
minimizing the intraclass scatter and maximizing interclass scatter simultaneously. Given a pairwise training set
" (x 1, y 1), f,(x i, y i), f,(x m, y m) ,, the objective function of
multiclass LDA to estimate the linear projection matrix P
can be written as follows:
	

max
P

tr(P T S b P)
, (16)
tr(P T S w P)

where S w and S b are defined as the within-class and between-class scatter matrices, respectively. With the constraint of P T S w P = I, the optimization problem in (16) can
be equivalently converted to one of S b P = mS w P by introducing the Lagrange multiplier m. The close-form solution
to the simplified optimization problem can be deduced by
generalized eigenvalues decomposition (GED).
Due to the sensitivity to complex, high-dimensional
noises caused by environmental and instrumental factors and the availability of labeled samples, the original
LDA inevitably suffers from an ill-posed statistical degradation, especially in the case of small-scale samples.
The degraded reasons mainly lie in the singularity of the
two scatter metrics (S w and S b), thereby easily leading to
the overfitting problem. To improve stability and generalization, the regularized LDA was proposed by adding an l 2- norm constraint on S w, parameterized by c as
S reg
w = S w + cI. By replacing S w in (16) with the regularized
S reg
w , the solution in the regularized LDA can be still obtained by the GED solver.
Considering the local neighborhood relations between
samples in the process of model learning, LFDA breaks
through the bottleneck of those LDA-based methods by
assuming that the data are distributed in the nonlinear
70

manifolds rather than a homogeneous Gaussian space. For
this purpose, LFDA is capable of effectively excavating the
locally underlying structure of the data that lie in the real
world. Essentially, LFDA can be regarded as a weighted LDA
by locally weighing S w and S b matrices. Therefore, the two
modified scatter matrices, denoted as Su w and Su b, can be formulated as

	

m

m

m

m

1
Su w = 2 / / W wij (x i - x j)(x i - x j)T,
i=1 j=1
1
Su b = 2 / / W bij (x i - x j)(x i - x j)T,
i=1 j=1


(17)

where the two weights (W w and W b) denote the samplewise similarities. There are several commonly used strategies
for calculating such a similarity matrix symbolized by W.
A simple yet effective one is given by W ij = 1, if x j ! z k(x i),
where z k (x i) represents the k-nearest neighbor of x i ; otherwise, W ij = 0. Another commonly used technique was
constructed based on the radial basis function with a standard derivation of v, as defined in (15). Refer to [62]-[64],
which might be useful for those who are interested in more
types of W.
Similar to SLDA, which first projects the original data
into a subspace and then LDA is performed in the transformed subspace, FSDA starts with maximizing the between-spectral scatter matrix (S f ) to enhance the differences along the spectral dimension; similarly, LDA is further
used to extract the representations of class separability
from the feature domain. In the first step, let n i, j be the average value of the jth class and the ith spectral band. Then,
we have the definition of S f as follows:
p

	

1
S f = 2 / (h i - hr )(h i - hr )T, (18)
i=1

where h i = [n i, 1, n i, 2, f, n i, k] is the spectral representation
p
in the feature space and hr = (1/p) / i = 1 h i . The primary
transformation (Pf ) that aims at improving spectral discriminant can be estimated by maximizing the trace term
of S f as
	

max tr (P Tf S f Pf ). (19)
Pf

Using the obtained Pf , the latent representation in the feature space g i = P Tf h i, i = 1, 2, f, p can be further fed into the
next step, LDA.
GDA AND ITS EXTENSIONS
Before revisiting the GDA methods, we first introduce and
formulate the general graph embedding (GGE) framework
presented in [49] with (12). Obviously, the extracted features Z in the GGE framework are determined by the construction of W to a great extent. Thus, we highlight several
types of representative affinity matrices corresponding to
the different graph-embedding approaches, i.e., LDA, LE
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

DECEMBER 2020
IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020
