IEEE Geoscience and Remote Sensing Magazine - December 2020 - 69

geometric nonlinear FE. ISOMAP searches for geodesic distances between data points and includes three main steps:
1) constructing a neighborhood graph of the data points,
2) computing the shortest path distances between all data
points in the neighborhood graph, and 3) creating the lower-dimensional embedding vectors that preserve the path
distances in the neighborhood graph.
Locally linear embedding (LLE) [46], Laplacian eigenmaps [47], and locality-preserving projection (LPP) [48]
are also geometric nonlinear FE techniques based on
graph embedding. LLE constructs the embedding graph
in three steps. First, the neighbors for data points are selected using the K nearest neighbors. Second, the weights
A i, j that linearly reconstruct the data points are computed
using their neighbors by minimizing the following constrained least squares:
	

n

min / x i A

i=1

/ A ij x j

j ! zi

2

s. t.

/ A ij = 1,(10)

j ! zi

2

where z i (x i) contains the neighborhood pixels selected for
x i . We should note that the constrained weights estimated
from (10) for every data point are invariant to rotations,
rescalings, and translations of that data point and its neighbors; therefore, they characterize the intrinsic geometric
properties of each neighborhood. Third, the lower-dimensional embedding vectors y are constructed by minimizing
n

/ zi min
z
	

i=1

/ A ij z j

j ! zi

2

s. t.

n

/ z i = 0,

i=1

2



n

1/
T
n i = 1 z i z i = I.

(11)

We should note that the reconstruction weights A ij are fixed
in minimization (11), and, therefore, the intrinsic geometric properties of the data with dimension p are invariant to
such a transformation into a lower-dimension d.
In [49], a general framework for graph embedding is
given by
	

min
z

/

i, j ! z i

z i - z j 22 W ij

s.t.

ZBZ T = I (12)

or, equivalently,
	

min tr (Z (D - W) Z T) = min tr (ZLZ T)
Z

Z

s.t. ZBZ T = I,



(13)

where L = D - W denotes the Laplacian matrix of the undirected weighted graph G = " X, W , (where X is the vertex
set and W ! R n # n is the similarity matrix) and D is a diagonal matrix where its entries are given by
	

D ii = / W ij, 6i. (14)
j!i

The diagonal matrix B is for the scale normalization and
might also be the Laplacian matrix of a penalty graph, such
DECEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

as G p = " X, W p ,. We should note that the vertices of G p and
G (i.e., X) are the same, while the similarity matrix (W p) corresponds to the similarity characteristics suppressed in the
lower-dimensional feature space (B = Lp = D p - W p; see [49]).
LLE can be reformulated using the graph embedding mentioned earlier with similarity matrix W ij = A ij + A Tij - A Tij A ij if
j ! z i; otherwise, W ij = 0 and B = I [49]. ISOMAP, LE, and
LPP can also be formulated using graph embedding [49].
From the viewpoint of graph embedding, the main difference between these FE techniques is the selection of the
matrices W and B. For instance, LE and LPP use the Gaussian function with the standard deviation v to choose the
similarity matrix as
	

W ij = * exp
0,

- < x i - x j < 22
, 6i, j ! z i (x i) (15)
2v 2
otherwise.

We should note that the techniques categorized in this
group are assumed to be SFE methods when they are applied only on the training samples. This is common in the
case of HSI due to the large volume of the image, which
makes the algorithm computationally very expensive. In
the following section, we discuss how the ground reference
(training samples) can be used to construct the edge matrix,
W; therefore, those techniques are considered SFE.
SFE TECHNIQUES
Unlike UFE techniques that rely on modeling various prior assumptions of hyperspectral data, supervised methods are capable of extracting class-separable features
more effectively, owing to the use of label information.
Over the past few decades, some seminal models have
been widely developed and applied to perform SFE on
HSIs; these can be roughly categorized into two streams:
subspace-learning (SL)-based and band-selection (BS)based approaches.
Different from handcrafted features [50], SL-based approaches learn to extract the low-dimensional representation from the data by formulating different supervised
rules in view of label information. There are some typical
methods in SL, including LDA [51], matrix discriminant
analysis (MDA) [52], decision boundary FE [53], and so
on. The latter BS-based methods, which aim at screening
out the representative and informative spectral bands,
are unfolded with MI-based BS [54], rough set, and fuzzy
C-means [55], to name a few. To further enhance class
separability, many extended methods have been successfully proposed in recent years: subspace LDA (SLDA) [56],
regularized LDA [57], local Fisher's discriminant analysis
(LFDA) [58], feature space discriminant analysis (FSDA)
[59], rough-set-based BS [60], and FE with local spatial
modeling [61].
Because of the powerful learning ability of SL methods compared to that of BS-based strategies, we focus
on reviewing the SL-related FE techniques, in which two
main streams-discriminant analysis FE (DAFE) and
69
IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020
