IEEE Geoscience and Remote Sensing Magazine - June 2019 - 164

inspiring work of [31] to compute end-members by benefiting from the correlation between spatially close pixels and of
[32] by exploiting a mixture of spatial and spectral classifiers.
More recent approaches focus on statistical models able to
learn directly over local neighborhoods (fixed or adaptive)
how to extract combined spectral and spatial features. In
particular, [33] introduced the possibility of designing spatial-spectral kernels for SVMs able to handle hyperspectral
data. This technique is adopted in later works [34]-[36]. With
a similar objective, [37] proposes a method to automatically
choose the filters that lead to the most efficient features for
hyperspectral data classification from a random filter bank.
The main limitation of traditional shallow-learning
methods stems from the feature engineering required to improve the classifier's performances. Indeed, spectra from different classes have to be separated in the feature space, which
can be challenging to achieve. In comparison, deep learning
is focused on representation learning, i.e., automatically designing a feature space that is tailored to the objective task.
This significantly reduces the need for feature engineering
and should hopefully improve performances because both
representation and classification will be jointly optimized.

models could efficiently model both long-range and shortrange dependencies in the spectral domain. A similar approach using RNNs treating hyperspectral pixels as a sequence of reflectances was introduced in [44], including a
pseudolabeling scheme for semisupervised learning. Finally, an approach using both the recurrent and convolutional
aspects was proposed by [45], in which the filtered features
were processed by recurrent layers.

DEEP LEARNING FOR HYPERSPECTRAL DATA
Recent works use deep-learning techniques for classifying
hyperspectral images. The review that follows is organized
to identify the main families of methods.

THE 1D OR 2D CONVOLUTIONAL NEURAL NETWORKS
A more recent approach is grounded in CNNs, which are
popular in multimedia vision. In computer vision, most
CNNs are designed using a first part that is convolutional
and performs the feature extraction and representation
learning and a second part that is fully connected and performs the classification. However, the number of filters is
proportional to the number of input channels, e.g., for a
first convolutional layer with a 5 # 5 kernel and n output
channels, there will be 5 # 5 # n # 3 for an RGB image
(three channels) but 5 # 5 # n # 100 for common hyperspectral images (with 100 bands). Therefore, a popular approach to transpose deep convolutional networks to HSI
consists of reducing the spectral dimension to close the gap
between hyperspectral and RGB images.

PREPROCESSING AND NORMALIZATION
The preprocessing and normalization processes used for
deep learning are similar to the ones used for standard machine learning. However, it is worth noting that most works
do not use band selection or saturated spectrum removal
but, instead, rely on the robustness of neural networks.
For unmixing, a now standard, unsupervised approach
[38], [39] consists of a network with two stacked autoencoders: the first one is used for denoising, while the second one
does the actual unmixing by enforcing a sparsity constraint.
SPECTRAL CLASSIFICATION
SUPERVISED LEARNING
The most straightforward evolution from shallow machine
learning to deep learning is using a deep fully connected
network instead of a standard classifier (SVM or random
forest). The principle remains the same, but the network
may thematically model the task in a finer way and with a
better discrimination capacity. This has been implemented
since the 2000s [40], [41] with small networks, and it was
brought up to date recently by [42] with unidimensional
CNNs, which learn a filter collection to be applied on individual spectra. However, processing sequential data can
also be done using recurrent neural networks (RNNs).
RNNs use memory to retrieve past information and are often used to process time series. [43] suggested using these
RNNs to classify hyperspectral data by assuming that these
164

UNSUPERVISED LEARNING
One of the most important benefits of deep learning for
processing hyperspectral data is the introduction of autoencoders. Indeed, the band-selection problem and, more generally, dimensionality reduction can be considered a data
compression issue. Within this perspective, autoencoders
allow for the learning of a smart compression with minimal
information loss, which is more efficient than, for example,
a standard PCA. Thus, [46] and later [47] proposed dimension reduction through a cascade of autoencoders for denoising followed by classification with a simple perceptron.
SPATIAL-SPECTRAL APPROACHES

SUPERVISED LEARNING
Thus, several works have proposed a CNN for hyperspectral
data classification. [48] used a PCA to project the hyperspectral data into a three-channel tensor to then perform
the classification using a standard 2D CNN architecture, as
shown in Figure 5. The architecture alternates convolutions
and dimension reduction (either by PCA or by sampling)
followed by a multilayer perceptron for the final classification step, as shown in Figure 5, with 2D convolutions in red
and 1D fully connected layers in blue. As an alternative, [50]
flattened the spatial dimensions to produce a 2D image with
a different shape, instead of a hypercube, and then applied
a traditional 2D CNN on the resulting image. One drawback of these methods is that they try to make hyperspectral
images similar to RGB ones, i.e., to force hyperspectral data
into the multimedia computer vision framework. However,
the specific properties of HSI might be wasted when doing
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

JUNE 2019

IEEE Geoscience and Remote Sensing Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2019