IEEE Geoscience and Remote Sensing Magazine - September 2020 - 78

ALGORITHMIC DEVELOPMENT
Therefore, significant development is still needed to achieve
global LCZ mapping because of the lack of high-quality labels and transferable classifiers for worldwide deployment.
There are various promising classifiers for LCZ recently proposed by different research groups, including RF, support
vector machines (SVMs) [30], canonical correlation forests
[31], [32], rotation forests [21], gradient-boosting machines
[33], and ensembles of multiple classifiers [34]. The data
used are mainly satellite data in the optical and microwave
ranges, such as Landsat, Sentinel-1, and Sentinel-2 images.
Recently, fusing of multisource data, such as satellite
images and Google Street View, has also been investigated
for LCZ classification [35]. Deep learning certainly played
an important role in LULC using remote sensing data [36].
Multiple algorithms based on convolutional neural networks, such as residual neural network and ResNeXt, [35],
[37]-[42] have been developed. These approaches provide
satisfying results for specific areas. However, according to
[8], [26], and [43], regional variations in vegetation and artificial materials as well as significant variations in cultural
and physical environmental factors cause large intraclass
variability of spectral signatures. One existing effort to further improve LCZ classification results is developing more
robust machine-learning models with high generalization
ability to facilitate efficient upscaling in a reasonable time
frame [27], [43]. Deep-learning-based models have been
shown to have better generalization ability; thus, they can
be better exploited for LCZ classification [36], [40].
Despite active algorithmic development, the global
transferability of a machine-learning LCZ model requires a
large quantity of globally distributed and reliable reference
data as a first step. Such a data set is nonexistent in the community; this task is addressed in this article.

CONTRIBUTIONS OF THIS ARTICLE
THE DATA SET
To answer the pressing need for LCZ training data sets, we
carefully selected and labeled 42 urban agglomerations
plus 10 additional smaller areas across all of the continents
(except Antarctica) around the globe. Their geographic distribution can be seen in Figure 2. Many polygons in those
cities were manually labeled by the authors. By projecting
these labels to the corresponding coregistered Sentinel-1
and Sentinel-2 images, we obtained 400,673 pairs of corresponding Sentinel-1 SAR and Sentinel-2 multispectral image patches with LCZ labels. An impression of the Sentinel
image patch pairs in the data set can be seen in Figure 3.
However, the actual patches in the data set have a dimension of 320 × 320 m, which is smaller than the visualization
in Figure 3. Accompanying this article, we provide open
access to this high-quality So2Sat LCZ42 data set to the research community. This is meant to foster the development
of fully automatic classification pipelines based on modern
machine-learning approaches and support the accelerated
use of LCZ mapping at global scale.
IMPROVED LABELING WORKFLOW
We found that merely following the definition of LCZs in
[9] and the labeling process mentioned in WUDAPT is not
optimal for a joint labeling activity by a group of people because of the vague definitions of some LCZ classes. To ensure the highest possible quality of the result, we designed
a rigorous labeling workflow and decision rules, shown
in Figure 4 and "Decision Rule of Local-Climate-Zone Labeling," respectively. Meetings were conducted before and
during the labeling process to calibrate our understanding
of the definitions of the 17 classes. Afterward, the labeling

FIGURE 2. The locations of the 42 main cities (green dots) and 10 additional cities (orange dots) included in the So2Sat LCZ42 data set.
(Source: Microsoft Bing Maps.)

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

SEPTEMBER 2020

IEEE Geoscience and Remote Sensing Magazine - September 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2020