Computational Intelligence - May 2013 - 46

To check the mapping induced by the similarity matrices,
we show in Table 1 the similarity matrix for the product aggregation of the dimension-wise similarity measures of the
valence, activation, and dominance scales. The location of
the maximum of each row (bold) shows the final translation
from the larger vocabulary (rows) to the smaller vocabulary
(columns). The most glaring error is that "fearful" is not in the
range of the mapping from large vocabulary to small vocabulary due to relatively low similarity to any word in the blog
mood vocabulary. Cases where one would expect to have a
mapping to "fearful" (e.g., "anxious," "stressed") do show ele-

Spanish to IEMOCAP Translation Performance
0.8

0.6

0.4

0.2

0.0

Sum/Avg Aggregation
Product Aggregation
Sum w/Valence and Activation
Product w/Valence and Activation
Linguistic Weighted Average
VSM Similarity

Jaccard Similarity

Subsethood

Figure 6 Performance of translating from the Spanish emotion
vocabulary to the categorical emotion vocabulary, which was the set
of emotion labels used for annotating the IEMOCAP corpus [52].

0.5

Spanish to LiveJournal Translation Performance

0.4

0.3

vated similarity to "fearful" but "angry" or "disgusted" are
higher. The observation that most of the values in the "fearful"
column are lower than the other columns, we normalized each
column by its maximum value. Doing this does in fact produce
the intuitive mapping of "anxious" and "stressed" to "fearful,"
but also changed other values.
To better quantify the intuitive goodness of the mapping
from one vocabulary to another, we undertook an evaluation
based on human performance on the same mapping task. We
found that at least one of the subject's choices matched the
predicted mapping except in the following five cases (i.e., performance of approximately 84%): "confused," "busy," "anxious,"
"hungry," and "hopeful." Filtering out clearly nonemotion
words like "hungry" may have improved the results here, but
our aim was to use a possibly noisy large vocabulary, since the
data came from the web.
To see if the fuzzy logic approach agreed with a simpler
approach, we converted the survey interval end-points to single
points by taking the midpoints of the subjects' intervals and
then averaging across all subjects. As points in the 3-D emotion
space, the mapping performance of Euclidean distance was
essentially the same as those determined by the fuzzy logic
similarity measures. However, a simple Euclidean distance metric loses some of the theoretical benefits we have argued for, as
it does not account for the shape of the membership functions
and cannot account for subsethood.
Based on the membership functions from the Spanish survey
and the previous English surveys, we constructed similarity
matrices between the Spanish words as input and the English
words as output. The similarity matrix of the Spanish words and
the Emotion Category Word vocabulary are shown in Table 2.
Overall, the best performance of 86.7% came from mapping
from the Spanish vocabulary to the Emotion Category Word
vocabulary using similarity (rather than subsethood), and aggregating the scale-wise similarities using the multiplicative product
of the three scales. The performance of mapping from Spanish to
the Blog Mood vocabulary was worse that with the Emotion
Category Word vocabulary as output because the much larger
size of the Blog Mood vocabulary resulted in more confusability.
The best performance for this task was 50% using similarity and
linguistic weighted average for aggregating the similarities. A
comparison of the different similarity and aggregation methods
can be seen in Fig. 6 for mapping from Spanish to the Emotion
Category Word vocabulary and Fig. 7 for mapping from Spanish
to the Blog Moods vocabulary.

0.2

B. Propositional Model (Model 2)
0.1

0.0

Sum/Avg Aggregation
Product Aggregation
Sum w/Valence and Activation
Product w/Valence and Activation
Linguistic Weighted Average
VSM Similarity Jaccard Similarity

Subsethood

Figure 7 Performance of translating Spanish emotion words to liveJournal mood labels (colloquial emotion words).

IEEE ComputatIonal IntEllIgEnCE magazInE | may 2013

For the propositional model, we collected a set of 1228 question-answer pairs from 110 human-human EMO20Q
matches, in which 71 unique emotion words were chosen. In
these matches, the players successfully guessed the other players' emotion words in 85% of the matches, requiring on average 12 turns.
In the set of question-answer pairs there were 761 unique
answer strings. We selected a set of 99 answers based on

Table of Contents for the Digital Edition of Computational Intelligence - May 2013