IEEE Geoscience and Remote Sensing Magazine - December 2020 - 77

together to make full use of their respective advantages. In
the past few years, numerous works have been proposed in
this direction.
One type of integration method is to use each network
independently and then combine their results together
[119], [121]-[123]. In [123], a parallel framework was proposed to extract spectral-spatial joint features from HSIs.
In this framework, SAE was employed to extract spectral
features of each pixel, and CNN was used to extract spatial
features from the corresponding image patch. These two results were fused by a fully connected layer. Similar to this
article, Xu et al. [121] also adopted the parallel framework
but used LSTM to extract the spectral features. In contrast,
Hang et al. [122] proposed a serial framework to fuse CNNs
and RNNs. Specifically, they used a CNN to extract the spatial features from each band of HSIs and then used an RNN
to fuse the extracted spatial features. In [119], Shi and Pun
also employed a serial framework to integrate the CNN and
RNN for spectral-spatial FE.
Another kind of integration method is embedding the
core component (i.e., convolutional operators) of CNNs
into AEs or RNNs [93], [116]. In [93], an unsupervised
spectral-spatial FE network was proposed. The whole
framework was similar to AEs, also adopting the so-called
encoder-decoder paradigm. However, the fully connected
operators in AEs were replaced by convolutional operators, so that the network can directly extract spectral-spatial joint features from cubes. In [116], Liu et al. proposed
a spectral-spatial FE method based on a convolutional
LSTM network. Instead of fully connected operators, they
also used convolutional operators in LSTM units. For a
given cube, each band was fed into the convolutional
LSTM unit sequentially. The convolutional operators could
extract the spatial features, while the recurrent operators
could extract the spectral features. The whole network was
optimized in an end-to-end manner, thus achieving satisfactory performance.
EXPERIMENTAL RESULTS
To evaluate the performance of different FE techniques,
we selected four techniques from the UFE category (i.e.,
PCA [19], multiscale structural total variation (MSTV) [39],
OTVCA [42], and LPP [48]), four techniques from the SFE
category (i.e., LDA [51], CGDA [74], LSDR [82], and JPlay
[87]), and five techniques from the deep FE category [i.e.,
SAE [124], RNN [122], CNN [125], convolutional AE (CAE)
[93], and convolutional RNN (CRNN) [116]]. Here, we set
the tuning parameters for those algorithms before representing the experimental results.
ALGORITHM SETUP
The parameter setting usually plays a crucial role in assessing the performance of FE algorithms. Subspace
dimension (or number of features, d) is a common parameter for all of the compared algorithms. Selection of
the number of features is a hard task for HSI analysis. The
DECEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

endmember selection/extraction, subspace identification, and/or rank selection are all referred to this subject
[126]-[128]. For a fair and simplified comparison, the
parameter d is assigned to be equal to the number of
classes (k). We should note that d in LDA is automatically determined as k - 1, due to the class separability
(Fisher's criterion).
UFE
◗◗ PCA: This method is a parameter-free technique.
◗◗ MSTV: In [39], all parameters are adjusted using a trialand-error approach. The multiscale parameters adjusting the degree of smoothness (as suggested in [39]) are
set to 0.003, 0.02, and 0.01. The spatial scale for the
structure extraction in three levels (as suggested in [39])
is set to 2, 1, and 3.
◗◗ OTVCA: This method is initialized as recommended in
[42]. The tuning parameter m, which controls the level of
smoothness applied on the features, is set to 1% of the
maximum intensity range of the data sets.
◗◗ LPP: The number of neighbors is set to 12. The bandwidth of the Gaussian kernel is set to 1.
SFE
A common strategy for model selection is to run cross validation (CV) on the training set, since the labeled samples
are available in SFE. Therefore, we used the CV strategy on
the following studied algorithms for parameter selection.
◗◗ LDA: This method can be viewed as a baseline for SFE.
There is no additional parameter in LDA.
◗◗ CGDA: Equation (23) can be tuned to a regularized optimization problem, where one extra parameter-regularized l 2- norm-must be set in advance in the process of
graph construction, which can be searched in the range
of {10-4, 10-3, 10-2, f, 10 1, 10 2} by CV. In the experiments, 0.1 is used for all three data sets.
◗◗ LSDR: Two parameters are involved in LSDR, the standard deviation for the Gaussian function and the regularization parameter, which are selected in the range of
{0.05, 0.1, f, 0.95, 1} and {10-2, 10-1, 101, 10 2}, respectively, using CV. Finally, v and m are both set to one in
our experiments.
◗◗ JPlay: There are three regularization parameters (a, b,
and c) that must be set in the JPlay model (32). With
the CV conducted on the training set of three different data sets, the regularization parameters are selected in the ranges of {10-2, 10-1, 101, 10 2}, yielding the
final setting of (a, b, c) = (0.1, 1, 1) for the first data
set, (a, b, c) = (0.1, 0.1, 1) for the second data set, and
(a, b, c) = (0.1, 1, 1) for the last data set.
DEEP FE
◗◗ SAE: The input of SAE is the original spectral information of each pixel. Three hidden layers are used. The
numbers of neurons from the first to the third hidden
layer are set to 32, 64, and 128, respectively. Rectified
77
IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020
