IEEE Geoscience and Remote Sensing Magazine - December 2023 - 76

expression. Zhan et al. [133] introduced this task in RS and
proposed a transformer-based model with two encoders,
an attention module, a multimodal fusion module, and
an object localization module. To evaluate their model,
the authors constructed a benchmark dataset consisting
of image-expression-bounding box triplets. Yu et al.
[134] proposed a novel method for semantic localization
in RS images. Unlike visual grounding, this task involves
identifying the most relevant position in a large-scale RS
image at the pixel level. The proposed method improves
the speed and accuracy of the task by utilizing a multilevel
likelihood expansion and a chaotic self-feeding mechanism
to enhance semantic frequency. The result is a more
efficient and effective approach for pixel-level semantic retrieval
in RS images.
EVALUATION METRICS
Evaluation of the output generated by vision-language models
can be done intuitively through human judgment. However,
this approach is expensive and time-consuming since it requires
a lot of manual effort, and it is also subject to user variability
due to the subjective nature of human judgment. As a result,
various automatic evaluation metrics have been developed to
assess the quality of generated output. In this section, an overview
of the metrics commonly used for evaluating different
vision-language tasks is provided. These metrics are grouped
into three categories: text generation metrics, retrieval metrics,
and image generation metrics.
TEXT GENERATION METRICS
The performance of text generation tasks, such as image
captioning and open-ended VQA, is usually evaluated using
standard text generation metrics that measure the similarity
between the model-generated text and the ground-truth text.
The definitions of the most common text generation metrics
are given in the following sections.
BILINGUAL EVALUATION UNDERSTUDY
The Bilingual Evaluation Understudy (BLEU) score [135]
is a widely used evaluation metric for language generation
tasks, originally developed for machine translation. BLEU
measures the degree of overlap between the generated sentence
and the corresponding human-written reference sentence,
using n-gram precision as the basis for comparison.
An n-gram is a sequence of one or more ordered words,
and n is typically set to one, two, three, or four. BLEU is
defined as follows:
BLEU BP expdn/ log
N
= $
where wn
wP
n=1
are positive weights such that R =1 n
n w ;1
N
=
Pn
is
the n-gram precision; and N is the maximum length of
n-gram, usually set to four. To account for the tendency
of shorter generated sentences to receive higher scores,
BLEU includes a brevity penalty (BP) term that penalizes
76
BP 05`
= .
M
c
U
j
(5)
where MU is the total number of matching unigrams, and c
is the total number of sets of unigrams in the ground-truth
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2023
nn
(1)
generated sentences that are shorter than the reference sentence.
This penalty term is calculated as follows:
BP = (
ec r
1
` -1 r j
, crif 2
c
, if #
(2)
where r is the length of the reference sentence, and c is the
length of the generated sentence.
RECALL-ORIENTED UNDERSTUDY
FOR GISTING EVALUATION
The Recall-Oriented Understudy for Gisting Evaluation
(ROUGE) [136] is a group of metrics initially used to evaluate
text summarization. ROUGE includes several types of metrics,
such as ROUGE-1, ROUGE-2, ROUGE-W, and ROUGE-L, each
designed for different tasks. Among these, ROUGE-L is commonly
used for text generation tasks. It measures the longest
common subsequence (LCS) between the generated sentence
and the reference sentence. While similar to BLEU, ROUGE-L
focuses on calculating the recall rate, whereas BLEU emphasizes
precision. The formula for ROUGE-L is as follows:
ROUGEL
=
RP 1 + b2
RP b2
lcslcs
lcslcs
+
b =PR
()
(3)
where Rlcs and Plcs are the recall and precision of the LCS
between the generated sentence and the reference sentence,
respectively, and
lcslcs
/.
METRIC FOR EVALUATION OF TRANSLATION
WITH EXPLICIT ORDERING
The Metric for Evaluation of Translation With Explicit Ordering
(METEOR) [137] is a popular evaluation metric that
addresses some of the limitations of the BLEU metric. Unlike
BLEU, which only considers n-gram precision, METEOR
computes the BLUE-1 score between the words of the reference
sentence and the generated sentence and then computes
both precision and recall on the matching results. This approach
makes METEOR more comprehensive than BLEU, as
it captures both the precision and recall rates of the generated
text. In addition to precision and recall, METEOR also incorporates
an explicit ordering component, which evaluates the
order of the matching n-grams between the generated sentence
and the reference sentence. This feature makes METEOR
more robust than BLEU in handling differences in word
order between the generated sentence and the reference. The
METEOR score is computed as the harmonic mean of the
precision and recall. The formula for METEOR is
METEOR RP ()PR j 1 - BP
= ` 10
+ 9
(4)
where P is the precision, R is the recall, and BP is defined
as follows:
IEEE Geoscience and Remote Sensing Magazine - December 2023

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2023
