IEEE Geoscience and Remote Sensing Magazine - September 2022 - 258

set. They select data points for which the model is the most
uncertain about. They split into conventional interclass uncertainty
heuristics and more recent epistemic uncertainty
heuristics. Interclass uncertainty heuristics select points
for which the model hesitates between two classes. They
include breaking ties [21], prediction entropy [22], or variation
ratios [23], which are highly related to margin-based
heuristics such as margin sampling [11], [24] or multiclass
level uncertainty [11], [25]. Margin-based heuristics compute
distances between data points and support vector machine
(SVM) [26] hyperplanes. Those heuristics are notably
relevant to refine decision frontiers between similar classes
with high intraclass variability. They are not suited, however,
to measure epistemic uncertainty, i.e., uncertainty
in little-known data space regions. An example of early
epistemic uncertainty heuristic is the query by committee
method [10], [27], which measures the disagreement between
a committee of models. It is based on the fact that
predictions of independent classifiers diverge in regions
without training data. In the remote sensing community,
the multiview disagreement approach [28] uses a committee
of classifiers trained on uncorrelated views from the feature
space (subparts of the feature space). The intuition of
multiview is the same as dropout [29] for neural networks:
it prevents the coadaptation of feature detectors. Finally, recent
advances in variational inference [30]-[33] have paved
the way to modern Bayesian models and new epistemic
heuristics [13], [14], [34], [35] that fully exploit the capacity
of Bayesian models to know when they do not know,
as mentioned in [36]. Incidentally, multiview used with a
one-layer neural network can be interpreted as a Bayesian
approximation. It would indeed be a special case of dropout,
where the weights related to one view are zeroed out.
Dropout itself can be used to approximate Bayesian neural
networks [30] and is then referred to as a Monte Carlo (MC)
dropout. Uncertainty-based AL methods are usually criticized
for their high sensibility to outliers, which may lead
to nonrepresentative data sets.
REPRESENTATIVENESS-BASED AL
To circumvent the problems raised by uncertainty heuristics,
representativeness-based approaches try to query
points to fit the unlabeled data distribution to the labeled
one. Some schemes minimize the maximum mean discrepancy
(a measure of distance) between the distributions [37],
while other methods have recently considered AL as a Core
Set, or submodularity, [12], [38] problem. Core Set problems
aim at selecting a subset of the training data set while
inducing minimal performance loss. The idea behind those
approaches is to query points to have labeled points homogeneously
spread over the whole data space. Meanwhile,
techniques such as [39] and [40] try to fool a discriminator
that spots unlabeled points from labeled points in an adversarial
fashion. Those implicitly bridge the gap between
the labeled and unlabeled data distributions. Although
Core Set approaches need to compute distances between
data points, adversarial approaches benefit from minibatch
optimization of the discriminator to handle large data sets.
Finally, a cluster-based approach introduced by [41] queries
points so that every cluster in the data are represented by
enough training samples. In remote sensing, a land cover
nomenclature can be described in a tree shape with nested
subclasses. For instance, a dry vegetation subclass could lie
inside a larger vegetation class. The more branches the tree
has, the more finely the nomenclature distinguishes the
classes. The idea of the Hierarchical method is to query pixels
so that the training data set can accurately capture the
level of detail in the nomenclature.
PERFORMANCE-BASED AL
Performance-based methods query points that increase
classifier performances on the test set. Those techniques
include expected error reduction and variance reduction
[10] through approximations or closed-form formulations.
Those procedures estimate how much the labeling of new
samples reduces the prediction error on the test data set.
Recently, Konyushkova et al. [15] suggested training a regressor
to predict the expected error reduction based on the
state of the model and candidate samples. To circumvent
the difficulty of estimating the test error reduction, Baram
et al. [42] uses a reinforcement learning paradigm where
each sample is a resource to be chosen. After a resource, i.e.,
a sample, is chosen, it is labeled and added to the training
data set. A reward is then given according to the increase of
accuracy the new samples have allowed. This is a particular
setting of the multiarmed bandit problem with a varying
Start
Manually Label the
Initialize the
Initial Training Data Set
Query b Samples
From the Pool to Maximize
the Acquisition Function
Acquisition Function
Label Samples
According to
the Oracle
Set the Budget b
and the Number of Steps
Add New Samples to
the Training Data Set
Final Step?
End
FIGURE 1. The pool-based AL flowchart.
258
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE SEPTEMBER 2022
IEEE Geoscience and Remote Sensing Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2022
