IEEE Geoscience and Remote Sensing Magazine - September 2021 - 44

generalized into morphological APs through the seminal
work of Dalla Mura et al. [7].
Unlike MPs, APs employ attribute filters (AFs) [8], a
powerful class of connected morphological filters capable
of removing entire connected components (CCs) with respect
to arbitrarily defined attributes (e.g., geometric, statistical,
and so on), thus eliminating the size/shape limitation
of MPs. Furthermore, APs can be generated efficiently
through quasilinear [9] and parallel [10] algorithms as well
as tree-based hierarchical representation of their inputs
[11], thus equipping them with a high level of scalability-
an invaluable property in remote sensing, where gigapixel
images are becoming the norm. In fact, [12] presents an application
of AP at a terapixel scale!
It is thus not surprising that, since their introduction
10 years ago, a great number of AP-related publications
have appeared (4,540 on Google Scholar as of June 2020),
tackling various aspects of remote sensing image analysis.
Many among them have been dedicated to further improving
APs through a rich variety of extensions, focusing on
every single stage of their calculation.
Even though recent years have witnessed the overwhelming
proliferation of deep learning [13] (overshadowing,
performancewise, most nondeep feature extraction
methods), APs continue to withstand the test of time. In
addition to their aforementioned invaluable properties,
this is also due to their ability to perform even with limited
amounts of training data as well as their capacity to accommodate
the arbitrary modalities and challenging types of
images so often encountered in the remote sensing domain.
This article (a short version of which appeared at the
2018 International Conference on Pattern Recognition and
Artificial Intelligence) presents a survey of APs and contributes
to the state of the art in the following ways:
◗ We provide a comprehensive review of 10 years of advances
on APs by decomposing their calculation into
four stages and grouping the reviewed studies accordingly.
Evidently, this is not the first survey on APs [14],
[15]. Contrary to [14] from five years ago, this survey is
not limited by the vanilla definition of APs and their
focus on hyperspectral data, and, compared to [15], it
additionally addresses APs on partition tree structures,
threshold-selection techniques, and AP postprocessing.
◗ We present the results of an extensive series of classification
experiments with multiple real data sets intended
to measure the performance of the various reviewed AP
variants. However, the experiments have been conducted
for the first time in terms of connectivity type as well
as spectral quantization level.
◗ More importantly, we present the results of pixel-classification
experiments intended to measure not only the
performance of the various reviewed AP variants but
their generalization capacity as well. In the big data era,
generalization is a core and sought-after property of any
content description tool, one for which APs have been
criticized, as an image's tree representation contains
44
both training and testing elements [16]. Mixing training/testing
sets is, unfortunately, not uncommon with
deep learning either [17]. We underline this validation
malpractice, often encountered in the state of the art, and
propose a solution through an image's spatial subdivision
and independent tree construction.
◗ To promote reproducible research and ease generalization,
we also provide the first available open source library
for APs (Simple Attribute Profiles [SAP]), available
at https://gitlab.inria.fr/fguiotte/sap.
THE PRINCIPLE OF APs
APs are multilevel image-description tools obtained by
successively applying a set of morphological AFs [7]. Unlike
usual image filtering operators, which are directly performed
on the pixel level, AFs work on the CC level based on
the concept of image connectivity. In particular, AFs are applied
on CCs with regard to a predicate based on an arbitrary
statistical or geometric property thereof. Consequently, they
exhibit a higher level of flexibility with respect to operators
by reconstruction, which are severely limited by characterizing
only the size and shape of their input. This advantage
naturally extends to APs versus MPs as well [2], [7].
The generation of APs [7] from an input image can be
summarized as a four-step process (see Figure 1):
1) construction of the image's hierarchical tree representation,
where CCs are denoted as nodes
2) computation of one or more relevant attributes describing
the geometrical and statistical features from each
tree node
3) filtering the tree by preserving/removing nodes according
to their attribute values compared against predefined
thresholds
4) reconstruction of the image from the filtered tree.
Step one can be performed using different pixel connectivity
rules. For 2D images, four- and eight-connectivity are
the most common. Steps three and four can be implemented
for different attributes (with varying threshold values) to
finally produce a set of filtered images (by stacking them),
forming the APs.
More formally, according to the seminal work of [7],
given a grayscale image :, E Z2
XE Z "
" ,K
" c =,k k
z
k k=1
K
,
of APs on X is achieved by applying a sequence of AFs
based on a min-tree (i.e., attribute-thickening operators
)
and max-tree (i.e., attribute-thinning operators
1 ) as follows:
APXX XX XX
XX
KK
KK
where ()Xk
z
() (),( ), ,( ), ,( ),zc ,
cc ,
= " zz ff-11 1
-1(),( ),
(1)
denotes the filtered image obtained by applying
the attribute thickening z with regard to the threshold
k. A similar explanation is made for ().Xk
c
As observed,
the resulting ()AP X is a stack of ()K21+ images including
the original image, K filtered images from the thickening
profiles, and the other K from the thinning profiles.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE SEPTEMBER 2021
3 the calculation
https://gitlab.inria.fr/fguiotte/sap
IEEE Geoscience and Remote Sensing Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2021
