IEEE Geoscience and Remote Sensing Magazine - September 2022 - 270

other classes. Batch-BALD selected a huge portion of pixels
(up to 70%) for the grass-pasture, woods, and buildinggrass-trees-drives
classes. BALD selected more than half of
the pixels in the alfalfa, grass-pasture-mowed, hay-windrowed,
and stone-steel-towers classes. Here we draw the
reader's attention to the fact that those classes are present in
low proportions (0.4, 0.3, 4.7, 0.2, and 0.9%, respectively).
VAAL also selected large portions of alfalfa, grass-pasturemowed,
hay-windrowed, and stone-steel-towers. Core Set
notably selected pixels from alfalfa, corn, grass-pasturemowed,
oats, and building-grass-trees-drives, which are
also present in rather small proportions. Hierarchical-selected
pixels in approximately equal proportions in each
class. Finally, LAL selected large portions of alfalfa, grasspasture-mowed,
and hay-windrowed.
PAVIA UNIVERSITY
We experimented with two different settings on the Pavia
University data set:
1) Setting 1: every class is represented in the initial training
data set.
2) Setting 2: the gravel, trees, painted metal sheets, and
bare soil classes are not represented in the initial training
data set.
SETTING 1
First, we notice that the OA [see Figure 10(a)] is very high
from the start, and that methods show small deviations.
Breaking Tie, Hierarchical, and random sampling achieved
approximately the same results. Core Set (with subsampling)
slowly increased but reached the same performance
than previous methods after 10 steps. In order of decreasing
metrics, BALD, VAAL, LAL, and Batch-BALD performed
significantly worse (more than a 2% gap).
In terms of mIoU [see Figure 10(b)], Breaking Tie and
Core Set (with subsampling) demonstrated significant gains
against random sampling and Hierarchical after a few steps.
In order of decreasing metrics, BALD, LAL, VAAL, and BatchBALD
performed worse.
With regard to the proportions of queried pixels (see
Figure 11), random sampling homogeneously queried pixels
in each class. Breaking Tie queried a large proportion
of gravel but no pixels of painted metal sheet and shadows,
both of which can be very easily discriminated from
other classes, as we see in Figure 5. On the contrary, BatchBALD
queried half of painted metal sheet and shadows but
almost no pixels from other classes. BALD selected large
portions of painted metal sheet, bare soil, and self-blocking
bricks. VAAL selected large portions of trees, painted
metal sheet, and shadows. The samples selected by Core
Set are more spread out over the different classes. It queried
few portions of meadows, trees, and barely any pixels
from painted metal sheet and shadows. Hierarchical had
the same behavior as random sampling. LAL took pixels
from meadows, trees, painted metal sheet, bare soil, and
shadows in small proportions.
270
SETTING 2
Considering the OA [see Figure 12(a)], Core Set (with subsampling)
followed the accuracy of random sampling for
the first six steps. Then it reached a better metric (between 1
and 2% higher). Breaking Tie, Hierarchical, and VAAL
started from a much lower accuracy and slowly reached the
performance of random sampling. BALD started as high as
Core Set but did not lead to significant improvements afterward.
Batch-BALD and LAL performed worse.
With regard to mIoU [see Figure 12(b)], Core Set (with
subsampling) demonstrated a significant gain over all the
other methods from beginning to end. Breaking Tie was
also better than random sampling, while Hierarchical and
VAAL showed the same results as random sampling. In decreasing
order of accuracy, BALD, LAL, and Batch-BALD
performed worse.
With respect to the proportions of queried pixels (see Figure
11), random sampling homogeneously queried pixels in
each class. Breaking Tie behaved like in setting 1, except that
it queried pixels from painted metal sheet, mostly at the first
step as it was not labeled in the initial data set. It selected the
same portion of trees but only at steps 2 and 3. Batch-BALD
behaved quite the same as in setting 1. BALD showed little
difference between the two settings as did as VAAL, Core
Set, and Hierarchical. LAL, compared to setting 1, queried
no pixels from trees but selected pixels from bare soil.
MAUZAC
The Mauzac data set exhibits higher complexity due to its
large size, higher spatial resolution, and land cover heterogeneity.
More particularly, we do not know the labels of the
queried pixels in advance (we labeled the pixels ourselves).
Therefore, there is no guarantee that queried pixels belong
to classes of the initial training data set. Annotating pixels
is difficult, time consuming, requires field campaigns, and
therefore expensive. Labeling the pixels for one query (one
method and one step) took, on average, a bit more than 1 h.
Thus, annotation for the seven schemes over five steps took
approximately 40 h (including a field campaign completed
beforehand that took half a day). For those reasons, we could
not run more than five steps per method. Besides, we see
that it is enough to draw conclusions. As for the previous
data sets, we depict the number of queried pixels per class
and we compute the OA [see Figure 13(a)] and mIoU [see
Figure 13(b)] for each step. At steps 3 and 5, approximately
0.475 and 0.5% of the pixels, respectively, are labeled.
Core Set and Hierarchical (applied with our preclustering
method), as well as Breaking Tie and BALD, significantly
outperformed random sampling (a more than 20%
higher OA) [see Figure 13(a)]. Core Set demonstrated a
quick increase but stagnated after step 3. On the contrary,
Hierarchical and BALD showed small improvements at the
first steps but a large increase at step 5. VAAL, although
better than random sampling, showed no improvement
during the course of the process. LAL achieved a much
worse accuracy.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE SEPTEMBER 2022
IEEE Geoscience and Remote Sensing Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2022
