IEEE Geoscience and Remote Sensing Magazine - June 2019 - 170

dimensions of the spatial kernels. Higher-resolution data
sets, such as the DFC2018, would probably benefit from
larger receptive fields to model long-distance spatial relationships between pixels.
PERSPECTIVES
Several potential research axes appear for the future. First,
following the works described previously, is the implementation of fully 3D, end-to-end networks for hyperspectral
data. Indeed, they are theoretically able to perform the
reduction dimension in both the spatial and spectral domains. However, because of the small spatial resolution,
a direct transfer of the 2D CNN, which mainly relies
on the spatial context of a full spectrum, is not the most
promising alternative. On the contrary, the transfer of 3D
FCNs could extract meaningful local features and tackle
this issue.
However, there is also a need for new, larger, and more
complex (e.g., with more classes) reference hyperspectral
data sets for the classification task. Indeed, the community
has developed standard approaches that already reach satisfying results on most of them, so it is no longer possible to
distinguish between the newly proposed approaches.
A second promising research field is the study of unsupervised and semisupervised models to overcome the
sparsely available annotations of hyperspectral data. For
example, 3D spatial-spectral autoencoders might learn
representations of hyperspectral scenes independent of
the acquisition context, which can be used for subsequent
tasks, such as clustering or classification.
Finally, a coming research field is data synthesis, which
would allow for simulation of realistic hyperspectral scenes.
In recent years, generative models, such as Gaussian mixture
models (GMMs) or generative adversarial networks (GANs),
have been applied to generate new training samples. Initially, the idea was that generative models could approximate
the distribution of deep features in the embedded space.
Then, the models infer new samples that could be used to
train larger classifiers. [79] (using GMMs) and [80] (using
GANs) are two examples of such works. However, thanks to
the theoretical improvements to the GAN framework, deep
generative models are now able to synthesize hyperspectral
pixels [81] and even small hyperspectral cubes using a 3D
CNN [82] from scratch. This can, of course, be used for data
augmentation, although the benefit of adding new data estimated from a generative model does not seem huge at this
point. Nonetheless, generative models will likely be a great
asset to estimate image transformations that are usually
costly to implement or require specific expert knowledge,
such as atmospheric correction, transfer function estimation between sensors, or image denoising.
CONCLUSIONS
Deep learning has already proven to be efficient for hyperspectral data. The 2D or 3D convolutional networks
allow for the intuitive combination of spatial and spectral
170

information, and the first published works show state-ofthe-art performances, often without expert knowledge
about the physics or the sensor.
Today, the main challenge is the scarce availability of
massively annotated data sets. Yet, data volume is the key
for the success of statistical approaches. A promising path is
the development of unsupervised approaches and data augmentation by synthesis to overcome this issue and unlock
the full potential of deep learning in HSI.
ACKNOWLEDGMENTS
We would like to thank Xavier Ceamanos and Naoto Yokoya
for their helpful and valuable insights on the various issues
examined in this article. We would also like to thank the National Center for Airborne Laser Mapping and the Hyperspectral Image Analysis Laboratory at the University of Houston for
acquiring and providing the DFC2018 data used in this study
and the GRSS Image Analysis and Data Fusion Technical Committee. We thank Prof. Paolo Gamba from the Telecommunications and Remote Sensing Laboratory, Pavia University (Italy) for providing the Pavia data set and Prof. David Landgrebe
and Larry Biehl from Purdue University, West Lafayette, Indiana, for providing the Indian Pines data set. Nicolas Audebert's
work is funded by the joint Total-Office National d'Études et de
Recherches Aérospatiales research project NAOMI.
AUTHOR INFORMATION
Nicolas Audebert (nicolas.audebert@onera.fr) received his
M.Eng. degree in 2015 from Supélec, Gif-sur-Yvette, France,
his M.Sc. degree in 2015 from Université Paris-Sud, France,
and his Ph.D. degree in computer science in 2018 from the
University of Bretagne Sud, Vannes, France. He is a research
scientist with the Information Processing and Systems Department at the Office National d'Études et de Recherches
Aérospatiales, University Paris Saclay, Palaiseau, France. He
was awarded the International Society for Photogammetry
and Remote Sensing Best Benchmarking Contribution prize
at the 2016 Geographic Object-Based Image Analysis Conference and Second Best Student Paper prize at the 2017 Joint
Urban Remote Sensing Event. His research interests include
the use of deep neural networks for scene understanding of
airborne and satellite optical images and Earth observation.
Bertrand Le Saux (bertrand.le_saux@onera.fr) received his M.Eng. and M.Sc. degrees in 1999 from the
Institut Polytechnique Grenoble, France, and his Ph.D.
degree in 2003 from the University of Versailles/Institut
National de Recherche en Informatique et Automatique,
Rocquencourt, France. He is a research scientist in the
Information Processing and Systems Department at the
Office National d'Études et de Recherches Aérospatiales,
University Paris Saclay, Palaiseau, France. He currently
serves as the chair of the IEEE Geoscience and Remote
Sensing Society Image Analysis and Data Fusion Technical
Committee. His research interests include visual understanding by means of data-driven techniques and developing machine-learning and deep-learning methods for
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

JUNE 2019

IEEE Geoscience and Remote Sensing Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2019