IEEE Geoscience and Remote Sensing Magazine - March 2021 - 71

with respect to the reality. The dictionary-based SR-D also
reaches overall very good performance (always very close
to the best VO approach for each test case), thanks to the
image self-similarity, which is exploited in the sparse representation. The results obtained by PWMBF often stay
below the other VO methods in terms of performance indexes for these tests. This can be associated with a slightly
greater spectral and spatial distortion with respect to the
other VO techniques.
MACHINE LEARNING
Let us start focusing on the two variants proposed in [91],
PNN and PNN-IDX. Contrary to the experimental observations provided in [91], here PNN-IDX does not provide
any performance gain over the baseline model PNN, giving even worse results on IK and GE-1. This can be justified by recalling that, in the original work, where PNN-IDX
was recognized as the best model, both of these versions
were trained on the same acquisitions used for tests (but
separate cuts), whereas, in the current experimental frame
train, test data are taken from different acquisitions. In
[95], the use of indexes as additional input bands was also
investigated employing different training and test acquisitions with results similar to ours. Therefore, we can safely
conclude that the use of those additional features in input
impacts the network robustness in real-world practical applications where the pretrained model has to be used as an
off-the-shelf tool.
Moving to the advanced models proposed in [95], APNN and A-PNN-FT perform better than the PNN and
PNN-IDX, on average. In particular, A-PNN achieves the
top performance on IK, not just among ML approaches,
whereas A-PNN-FT is the best on GE-1. On WV-3 images,
instead, the various versions of PNN appear to lose their
edge over non-ML methods. This is due in part to the fact
that these ML models were originally conceived and optimized for WV-2 images, while here we test their versions
retrained on a WV-3 data set of Rio de Janeiro.
At first glance, one may conclude that the FT is not as
effective as expected. Indeed, this is only partly true. We
may observe, in fact, that all models are pretrained by
downgrading the resolution of the data sets assigned to
the training phase. In addition to this, at inference time,
A-PNN-FT performs additional training iterations on the
RR version of the test image. As a consequence, if the test
is performed on an image whose resolution has already
been reduced (as in this evaluation framework), the downgrading applies twice, thus creating a resolution mismatch
between the pretraining and the FT stages. This penalizes
the FT version not only because of the poorer resolution
of the training sample but also because of its reduced size,
which can become very small if the test image is not sufficiently large. In our case, the FT eventually operates on
a 128 × 128 (PAN scale) input. Of course, what is really
important is that this mismatch does not occur in practical
applications at FR.
MARCH 2021

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

FULL-RESOLUTION PERFORMANCE ASSESSMENT
This section focuses on the experimental results at FR. The
comparison will be performed by splitting the section into
four subsections, one for each class. For a broader (interclass) discussion, interested readers can refer to the " Discussion " section, which completes the discussion of the
experimental results.
COMPONENT SUBSTITUTION
The analyses of the FR quality indexes in Table 4 are in substantial accordance with the results obtained at RR. However, the presence of details at a higher resolution gives evidence of different characteristics. In particular, it becomes
clear that the improvements caused by the segmentation
algorithms are mainly due to a significant reduction of the
spatial distortion. The most evident difference from the
RR protocol is in regards to the PRACS method, which obtains the best results in terms of the chosen quality indexes
without reference, rewarding the optimal balance between
detail enhancement and consistency with respect to the
original data.
In fact, as is also clear from Figure 9, most algorithms
perform strong enhancement of the local contrast entailing the presence of zones with unnatural colors (see, for example, the colors of the roofs in the C-BDSD pansharpened
product related to the Toulouse data set).
MULTIRESOLUTION ANALYSIS
The performance of the algorithms at FR strictly matches
those obtained at RR, underlining the significant improvement achievable by running the MTF-GLP-CBD on image
segments and by correcting the HPM scheme to take into
account the atmospheric effects.
Thanks to the presence of vast rural and vegetated areas,
the Collazzone data set highlights the importance of considering the path-radiance contribution. Figure 10(c), related to the MTF-GLP-HPM-H method, shows the significant
noise reduction obtained by excluding the haze from the
pansharpening process. The corresponding quality indexes
in Table 4 confirm the best results obtained by this method.
VARIATIONAL OPTIMIZATION-BASED METHODS
The results reported in Table 4 for the VO techniques overall consolidate the outcomes of the analysis at RR. Overall
good results can be pointed out by both TV and SR-D. In
particular, the TV is the best method for the WV-3 NY data
set thanks to its spatial regularization, which can positively
match the urban and human-made structures. SR-D also
obtains high-performance scores, overall outperforming
all other VO techniques for the Toulouse and the GE-1 data
sets. The results of the FE-HPM and PWMBF techniques are,
in general, behind those algorithms, as also shown at RR.
Finally, the RR method often shows the opposite behavior
with respect to the RR assessment (e.g., it was the best VO
method for IK Toulouse at RR but got the lowest performance for the related FR data set).
71

IEEE Geoscience and Remote Sensing Magazine - March 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - March 2021