IEEE Geoscience and Remote Sensing Magazine - December 2020 - 134

TECHNICAL COMMITTEES
NAOTO YOKOYA, PEDRAM GHAMISI, RONNY HÄNSCH, AND MICHAEL SCHMITT

Report on the 2020 IEEE GRSS Data
Fusion Contest-Global Land Cover
Mapping With Weak Supervision

he Image A nalysis and Data Fusion -Technical
Committee (IADF TC) of the IEEE Geoscience and
Remote Sensing Society (GRSS) has organized the
annual Data Fusion Contest (DFC) since 2006. The
contest aims to establish new benchmarks for scientific challenges in remote sensing image analysis by
promoting the use of multimodal data, leveraging
new sensors as well as big data, and facilitating the
emergence of new methods for extracting geospatial
information [1]-[3].
THE 2020 DATA FUSION CONTEST
The 2020 IEEE GRSS DFC aimed to cut through the
fundamental challenge of applying machine learning
to large-scale Earth observation image analysis: the
difficulty of obtaining high-resolution labeled data on a
global scale. The task was to perform large-scale land
cover mapping from globally available, multimodal
images with weak supervision, using low-resolution,
noisy labels as training data. Such technology will enable the automatic updating of land cover classification maps across the globe, with tremendous potential

Digital Object Identifier 10.1109/MGRS.2020.3033515
Date of current version: 16 December 2020

134

for a multitude of applications, including smart cities,
environmental monitoring, and disaster response.
In the contest, the SEN12MS data set [4] was available
to train land cover prediction models. The SEN12MS data
set includes Sentinel-1 synthetic aperture radar data, Sentinel-2 multispectral imagery, and moderate-resolution imaging spectrometer (MODIS)-derived low-resolution land
cover maps [5] sampled across the globe. All data have a
ground sampling distance (GSD) of 10 m, offer an image
size of 256 × 256 pixels, and contain more than 180,000
triplets from the entire globe. Ten land cover classes were
used for classification, each on a simplified version of the
International Geosphere-Biosphere Programme (IGBP)
classification scheme [6]. MODIS-derived land cover
maps have a resolution of 500 m per pixel, and the classification accuracy for the IGBP scheme is approximately
67% [7]. The key was how to effectively train a classifier
at a 10-m GSD from such noisy labels. We derived highresolution land cover maps semimanually for validation
and testing (Figure 1). The validation and test data were
not included in the SEN12MS data set and were distributed without geocoordinate information.
The 2020 DFC consisted of two successive tracks,
each addressing a slightly different problem. The objective of track 1 was to predict land cover labels using only
low-resolution labels for training and hyperparameter
tuning. In track 2, the task was to train the model for
land cover mapping using both low-resolution labels
and a limited number of high-resolution labels, which
were disclosed for the validation set after track 1. The
performance of both tracks was assessed using the average accuracy (AA) of all classes. The baseline approaches
were released as open source code.
The contest was divided into the following three phases:
1) Phase 1: On 13 December 2019, the validation data
without high-resolution labels were released at the
IEEE DataPort (https://ieee-dataport.org/competitions/
2020-ieee-grss-data-fusion-contest). On 13 January
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

DECEMBER 2020

https://ieee-dataport.org/competitions/2020-ieee-grss-data-fusion-contest https://ieee-dataport.org/competitions/2020-ieee-grss-data-fusion-contest

IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020