IEEE Geoscience and Remote Sensing Magazine - September 2020 - 85

Original Label
Final Label (After Majority Voting)

1
2
3
4
5
6
7
8
9
10
A
B
C
D
E
F
G

1
2
3
4
5
6
7
8
9
10
A
B
C
D
E
F
G

80 5
0 94
0 22
4 3
0 9
0 1
0 6
0 1
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 1
0 0

0
3
66
0
1
7
45
0
0
0
0
0
0
0
0
0
0

15
1
0
72
2
0
0
0
0
0
0
0
0
0
0
0
0

0
2
3
20
79
4
0
0
0
0
0
0
0
0
0
1
0

0
0
10
0
8
84
0
0
19
0
0
0
0
0
0
0
0

0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 4 0 0 0
45 4 0 0 0 0
0 97 0 1 0 0
1 1 79 0 0 0
0 11 0 87 0 0
0 0 0 0 98 1
0 0 0 0 5 88
0 0 0 0 0 18
0 0 0 0 0 0
1 1 0 0 0 0
0 1 0 0 0 0
0 0 0 0 0 0
1 2 3 4 5 6 7 8 9 10 A B
Label Validation Votes
(a)

80 5
0 89
0 5
5 3
0 7
0 1
0 7
0 1
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 1
0 0

0
4
70
0
1
6
32
0
0
0
0
0
0
0
0
0
0

15
1
0
76
3
0
0
0
0
0
0
0
0
0
0
0
0

0
4
1
15
85
5
0
0
0
0
0
0
0
0
0
0
0

0
1
5
0
4
86
0
0
14
0
0
0
0
0
0
0
0

0
1
18
0
0
0
58
1
1
0
0
0
0
0
1
0
0

0 0
0 0
1 0
0 0
0 0
0 2
2 0
95 0
1 84
6 0
0 0
0 0
0 0
0 0
1 0
1 0
0 0

0
0
0
0
0
0
0
2
0
93
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
97
3
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
2
89
17
1
0
0
0

0
0
0
0
0
0
0
0
0
0
0
7
62
2
1
1
0
C

0
0
0
0
0
0
0
0
0
0
0
7
66
3
1
1
0

0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 1
0 0
0 0
0 0
0 0
6 7
96 0
0 94
1 12
0 0

0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
6 0
1 0
3 0
83 0
0 100

D E F G

0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 1
0 0
0 1
0 0
0 0
3 7
95 0
0 95
1 12
0 0

0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
6 0
1 0
1 0
85 0
0 100

1 2 3 4 5 6 7 8 9 10 A B C D E F G
Label Validation Votes
(c)

0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0
45 1 0 0 0 0 0 0 0 0 0
0 96 0 2 0 0 0 0 1 0 0
0 0 67 0 0 0 0 0 0 0 0
0 13 0 86 0 0 0 0 1 0 0
0 0 0 0 98 1 0 0 0 0 0
0 0 0 0 4 88 7 1 0 0 0
0 0 0 0 0 23 49 23 1 4 0
0 0 1 0 0 0 0 98 0 0 0
0 1 0 0 0 0 0 0 87 12 0
0 1 0 0 0 0 5 1 14 80 0
0 0 0 0 0 0 0 0 0 0 100
1 2 3 4 5 6 7 8 9 10 A B C D E F G
Label Validation Votes
(b)

81 9 0
0 97 1
0 39 46
3 3 0
0 12 1
0 0 8
0 4 50
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0

81 9
0 95
0 4
3 4
0 10
0 1
0 5
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0

0
1
69
0
1
7
38
0
0
0
0
0
0
0
0
0
0

10 0 0
1 1 0
0 3 11
69 24 0
2 77 8
0 2 89
0 0 0
0 0 0
0 0 32
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0

10
1
0
74
2
0
0
0
0
0
0
0
0
0
0
0
0

0
2
0
18
82
3
0
0
0
0
0
0
0
0
0
0
0

0
0
4
0
5
88
0
0
15
0
0
0
0
0
0
0
0

0 0 0 0
0 0 0 0
23 0 0 0
0 0 0 0
0 0 0 0
0 0 1 0
56 1 0 0
0 96 0 3
0 0 83 0
0 11 0 88
0 0 0 0
0 0 0 0
0 0 0 0
0 0 1 0
0 3 0 0
0 0 0 0
0 0 0 0

0
0
0
0
0
0
0
0
0
0
98
3
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
1
89
20
1
0
0
0

0
0
0
0
0
0
0
0
0
0
0
7
64
1
1
3
0

0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 1
0 0
0 1
0 0
1 0
8 2
98 0
0 96
1 15
0 0

0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
6 0
0 0
1 0
80 0
0 100

1 2 3 4 5 6 7 8 9 10 A B C D E F G
Label Validation Votes
(d)

FIGURE 5. The confusion matrices (values as percentages) of the original and final labels (refined by majority voting) versus the votes cast by
the label validation crew for the polygons of the evaluation cities selected in Tables 2 and 3: (a) polygon-wise assessment of the original labels,
(b) pixel-wise assessment of the original labels, (c) polygon-wise assessment of the final labels, and (d) pixel-wise assessment of final labels.

before refinement by majority voting and 85% after refinement, the So2Sat LCZ42 data set can be considered a reliable
source of labels for the training of machine-learning procedures aimed at automated LCZ mapping on a larger scale.
BASELINE CLASSIFICATION ACCURACY
To provide a baseline for achievable LCZ classification accuracy, we performed classification on the So2Sat LCZ42
data set using popular classifiers, including classical RFs,
SVMs [30], and an attention-based ResNeXt, as proposed in
[47] and [48]. The employed RF consists of 200 trees, and
the max_depth is set to 10, with the other parameters set
to the default. A radial basis function kernel is chosen for
the SVM in the experiment. The depth of the ResNeXt is
SEPTEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

29 layers, and the convolutional block attention module is
plugged into each of the residual blocks. For RF and SVM,
the pixel values of the patches are converted into vectors,
using the statistical measures (maximum, minimum, standard deviations, and mean) of each band. All of the classifiers are trained using the training set and tested on the
validation set.
The resulting accuracy based on the Sentinel-2 images in
the So2Sat LCZ42 data set can be seen in Table 5. The accuracy measures include overall accuracy, averaged accuracy,
and the kappa coefficient. In addition, weighted accuracy,
introduced in [28], is considered, because it gives userdefined weights to confusions between different classes.
For example, misclassifying compact high rise as compact
85
IEEE Geoscience and Remote Sensing Magazine - September 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2020
