IEEE Geoscience and Remote Sensing Magazine - June 2019 - 168

that the SGD starting point has better convergence properties. The initialization policy from [74] is especially suited
for CNNs including ReLU activation. Also, because of their
large number of weights, the fully connected layers are often prone to overfitting. Dropout [75] significantly alleviates this phenomenon. Moreover, deeper networks often
benefit from a larger batch size and the use of batch normalization [76], which smooths the loss landscape.
Training sample preparation also includes a few practices that can be applied for better performances. First, shuffling the data set after each epoch helps avoiding recurring
patterns in the SGD and overall makes the optimization
smoother. Data augmentation is especially useful to introduce equivariances. For example, in image data, horizontal and vertical symmetries can be applied by flipping
the training patches randomly during the training. This
increases the diversity of the examples and the robustness
of the model. Moreover, many data sets present large class
imbalances, where one or a few classes dominate the labels.
One simple solution is to weight them according to the loss
function to penalize the less frequently occurring classes
more. Inverse-median frequency class weighting is commonly used to do so, e.g., in semantic segmentation. This is
equivalent to showing more examples from the rarer classes
to the model.
It is fundamental to be careful when tuning the optimization hyperparameters. Their choice should be based on
a validation set that is not the same as the test set, or the
test results will be optimistic. If this is not possible, a crossvalidation over several train and test splits helps to assess
how robust the hyperparameters are to avoid overfitting.
Finally, during inference, it is recommended to use the
network that reached the best validation score and not necessarily the last epoch weights. This implies saving regular
checkpoints during the training. We tried our best in our
toolbox to apply these best practices while letting advanced
users use their own parameters where needed.
EXPERIMENTS
In this section, we compare several deep architectures from
the literature for hyperspectral image classification in a remote sensing context. To the best of our knowledge, there

(a)

(b)

FIGURE 9. The train (light blue), test (dark blue), receptive field

(light green), and test pixel in the receptive field (dark green). The
dot indicates the center pixel. The 2D receptive field of a CNN can
involuntarily include samples from the test set, making the network
overfit and biasing the evaluation. (a) Random train and test and
(b) disjointed train and test.
168

have been no principled analyses of the various deep convolutional networks introduced in the past. Indeed, works
from the literature performed experiments using slightly
different setups:
◗ Most articles divided the data sets into train and test
splits by randomly sampling over the whole image. A
few articles [43], [77] used the standard train and test
split from the GRSS DASE initiative.
◗ Some authors considered only a subset of the classes.
This was especially prominent for Indian Pines, where
the classes with fewer than 100 samples were often excluded [42], [49].
◗ Even when the train and test splits are done the same
way, the number of samples in the training set might
vary. Some authors used 20% of all of the training set,
while others used a fixed amount of samples for each
(e.g., 200 samples for each class).
◗ Some authors further divided the training set into a
proper training set and a validation set for hyperparameter tuning, while others performed the tuning directly
on the test set.
In this article, we argue that randomly sampling the
training samples over the whole image is not a realistic use
case. Moreover, we affirm that it is a poor indication of generalization power. Indeed, neighboring pixels will be highly
correlated, which means that the test set will be very close to
the training set. To demonstrate this, we consider a nearestneighbor baseline using randomly sampled training pixels
and another using colocated training pixels well-separated
from the test set.
Also, in the case of 2D and 3D approaches, especially
CNNs, the receptive field of the network might involuntarily
include test samples in the training set. Indeed, the first convolutional layer also sees the neighbors of the central pixel,
which might be in the test set if the sampling has not been
carefully checked. An example of this is illustrated in Figure 9.
Following the standards for machine learning from
the remote sensing community, we performed our experiments using well-defined train and test splits where the
samples were extracted from significantly disjointed parts
of the image. In the case of 3D CNNs, it ensures that no
pixel from the test set will be surreptitiously introduced in
the training set. To do so, we used the train and test splits
for Indian Pines, Pavia University, and the DFC2018 as
defined by the GRSS on the DASE benchmarking website.
The ground truth is divided based on the connected components instead of the pixels, which allows for the evaluation to actually measure how the model generalizes to
new geoentities. Hyperparameters are tuned using 5% of
the training set as a separated validation set.
We used our toolbox to compare various reimplementations of the state of the art. Models were implemented as
closely as possible to the use in the original articles. It is worth
noting that, despite the our best efforts, reproducing the results exactly, using only the articles and without the original
implementations, is very difficult. Many hyperparameters
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

JUNE 2019

IEEE Geoscience and Remote Sensing Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2019