IEEE Geoscience and Remote Sensing Magazine - December 2017 - 81

come after the metadata fields. Each data line consists
of values separated by commas. The first number is the
class label; the rest are the spectral band values. The first
entries (lines) contain the training data. Below the training data come the test data, which have the same format
except for the class label, which is set to −1 (unknown).
USING THE DOWNLOADED DATA SETS
The second section explains in detail how to use the downloaded data sets and how to predict a class for every pixel
with an unknown label. The predicted labels must be provided in the same order as in the data set file; otherwise, the
system will not properly evaluate them during the testing
phase. The predictions must be written as plain text files.
These files are also structured in lines; each line includes
the name of the method employed for the prediction, the
name of the reference data set the user wants to predict, and
the class labels. All these fields are separated by spaces. As
for the data set files, we provide code examples illustrating
how to write the test files. Because a single line contains
the predictions for a given data set and method, users can
readily provide several method predictions in a single test
file submission.
UPLOADING PREDICTIONS
The third and last section allows users to upload predictions.
The system presents a standard file dialog window that allows

users to choose their prediction file and upload it. Once the
file is uploaded, the system makes a series of redundancy
and sanity checks to ensure
the validity of the file format:
valid names for the reference
USERS CAN USE THE
data sets and user methods,
SYSTEM TO IMPROVE
the correct number of providTHEIR ALGORITHMS AND
ed predictions, and the same
TO CLAIM RESULTS IN
number of classes as reference
THEIR RESEARCH
data sets, to name a few. After
PUBLICATIONS ON THIS
these checks, the system comLARGE, REPRESENTATIVE,
putes the confusion matrix,
INDEPENDENT DATABASE.
and, from this, it estimates the
overall accuracy, Cohen kappa
coefficient [11], confidence intervals, significance levels, and the user's and producer's accuracy per class. All these estimations are then presented to
the user and stored in the database for future reference and
comparison with other methods.
Finally, the system contains a "Hall of Fame" page (see
Figure 1) where users can compare their results against
those of other users and methods. For each data set, the
best ten results are ranked and displayed in terms of overall
accuracy and kappa statistics. By clicking on each row of
the ranking table, more details are presented, namely, the
whole confusion matrix and the user's and producer's accuracy per class.

FIGURE 1. The processing steps in the HyperLabelMe platform: download a data set (left), train your model and upload your results (upper

right), and get the classification results (lower right).
DECEMBER 2017

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

81
Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2017
