IEEE Geoscience and Remote Sensing Magazine - September 2021 - 50

by the size and shape of structuring elements. However, using
APs in remote sensing images also has a few limitations.
First, apart from TF APs [66], [74], [75], the quality of an
AP depends on the selected set of thresholds. A bad selection
can lead to redundant information in the AP [61], [62].
An alternative solution to alleviate this redundancy problem
is to replace the AFs used in the computation of APs by
extinction filters, resulting in EPs [91].
An extinction filter acts on the regional extrema (minima
or maxima) of an image: each extremum is either completely
preserved or pruned. Let X be a grayscale image. The extinction
value (with respect to a given attribute) of any maximum
M of X is the maximum attribute value k such that M is still
included in a maximum of (),Xk
c
where k
c is the thinning
operator with parameter k. Similarly, the extinction value
of any minimum M of X is the maximum attribute value k
such that M is still included in a minimum of (),Xk
z
k
where
z is the thickening operator with parameter k. This way, to
compute an EP, the filtering parameter is the number of minima
or maxima to be preserved instead of a threshold value,
which makes EPs less sensitive to image resolution [91].
Since [91], EPs and their extension to hyperspectral images
(extended EPs) have been successfully applied to the
land cover classification of hyperspectral data [49], [92]
and fusion of hyperspectral and lidar data [30]. To further
reduce redundancy, composite kernels are used to fuse the
spatial information of EPs with hyperspectral data in [93]
and with lidar data in [47].
Another limitation of APs is that, very often, clusters
of pixels associated with distinct semantic objects, such
as roads and buildings, are connected by narrow paths of
similar intensity value. This leads to pixels of different semantic
classes being connected throughout several levels of
a component tree. Consequently, the attribute values of several
CCS describe the union of objects of different classes
instead of an object of a single class. In [94], the authors
address those problems in the context of hyperspectral image
classification. To overcome those issues, attribute connected
filters are replaced by partial reconstruction filters,
which allows the disconnection of regions connected by
narrow paths and improves the overall classification accuracy.
This idea is explored in [43], which shows the interest
of using partial reconstruction in the classification of
hyperspectral and lidar images in comparison to attribute
connected filters.
Another related approach, called invariant APs (IAPs),
was recently proposed in [95] to overcome other limitations
of APs, including the sensibility of APs to geometric
transformations, like rotation, and to the surrounding of
pixels of a same material. Different from APs, IAPs are not
computed from a hierarchical representation of the input
data. Instead, hierarchical information is indirectly extracted
from the original image by performing convolutions of
different sizes and computing the Fourier transform for different
values of the Fourier order. Then, IAPs are obtained
by stacking spatially invariant profiles, acquired from the
50
segmentations of the convoluted images, and frequency invariant
profiles, acquired from the histogram of oriented
gradients of the Fourier transforms.
Finally, Pham et al. [96] propose a generalization of APs
called feature profiles (FPs). They generalize step four of the
generation of APs that consists of reconstructing an image
from a filtered tree. To build an AP, this reconstruction is
originally performed by projecting the gray values of the
nodes of the filtered tree onto the image pixels. In [96], this
reconstruction step is extended by taking into consideration
not only the gray values of the nodes but also other
statistical and geometrical features. The resulting images
compose the so-called FP. The experiments with remote
sensing images of [96] and [97] show the interest of projecting
attributes like area and moment of inertia in the context
of image classification. Hence, we also consider FPs later in
the " Experimental Study " section.
EXPERIMENTAL STUDY
This section describes our experimental study to evaluate
the performance of standard APs as well as some of their recent
variants. The contributions of this section are twofold:
the evaluation of 1) newer variants of APs and 2) the impact
of connectivity and quantization parameters on the performance
of APs. This is the first study addressing the effect of
connectivity and quantization parameters in this context.
Experiments were mostly performed in Python using
publicly available libraries. (Source codes are available at
https://gitlab.inria.fr/dsantana/attributes-profiles-survey
-source-codes.) First, APs and some of their variants were
computed with the SAP package. (The documentation and
source codes are provided at https://gitlab.inria.fr/fguiotte/
sap.) To the best of our knowledge, this is the first available
open source library for computing APs and some of
their extensions. The SAP package relies on the Higra [98]
library (the documentation and source codes are given at
https://github.com/higra), which provides efficient implementation
and postprocessing of morphological trees in
C++. Then, classification was performed with the scikitlearn
Python library.
Supervised classification has been conducted on both
grayscale and hyperspectral images for the sake of comprehensiveness.
In this section, we introduce the data
sets, experimental setup commonly encountered in the
state of the art [7], [14], and classification results that have
been obtained.
In more detail, the standard setup involves calculating
the hierarchical tree representation (as explained in detail
in the " Principle of APs " section) from the entire input image,
computing the APs (or their variants) from this representation,
and then subdividing the resulting features
based on the locations of training and validation/testing
pixels. The next section elaborates on the reasons that this
approach constitutes a validation malpractice and proposes
an alternative strategy for better assessing the generalization
capacity of APs.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE SEPTEMBER 2021
https://gitlab.inria.fr/dsantana/attributes-profiles-survey-source-codes https://gitlab.inria.fr/dsantana/attributes-profiles-survey-source-codes https://gitlab.inria.fr/fguiotte/sap https://gitlab.inria.fr/fguiotte/sap https://www.github.com/higra
IEEE Geoscience and Remote Sensing Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2021
