IEEE Computational Intelligence Magazine - February 2020 - 72

four systems motivates us to combine the
predictions for overall better performance.
Consequently, we train an MLP based
stacked ensemble on top of the best
performing individual models, i.e., one
each for CNN, LSTM, GRU and Feature-SVR. In response, the MLP ensemble
network reports enhanced scores for each
of the datasets as reported in Table V. We
obtain Pearson scores of 0.747, 0.712,
0.755 & 0.779, for 'anger', 'joy', 'sadness' and
'fear', respectively. The ensemble network
improves the performance of individual
systems by a significant margin of 4, 2, 5 &
5 Pearson scores, respectively. Similarly, the
proposed ensemble approach aids in
improving the performance of the individual systems by 3 & 2 cosine similarity
points at 0.797 and 0.786, respectively for
the microblog messages and news headlines.
We compare our proposed system
with state-of-the-art systems for both

closer to the desired or gold intensity
score) than the competing systems (B, C
& D), while for some other examples system A reports less accurate prediction than
the other systems.We depict the contrasting behavior of these competing systems
with respect to the gold score in Figure 3.
Further, we highlight a few scenarios in
Figure  3a to make the differences more
apparent. In the first highlighted region,
CNN performs better than other systems,
whereas, in the second case both CNN
and GRU report closer values to the gold
score. Subsequently, in the third region,
SVR is the best among all, however, in
the fourth region, it has the least performance. Similarly, SVR has the best and
least performances for the sixth and seventh highlighted regions, respectively. In
contrast, LSTM and GRU both have better performances for the fifth highlighted
region. Such contrasting behavior of these

scores of 0.709, 0.679, 0.674 & 0.707 and
0.706, 0.704, 0.701 & 0.748, respectively.
In microblog dataset four individual models, i.e., CNN, LSTM, GRU and featurebased systems obtain cosine similarity of
0.724, 0.727, 0.721 and 0.765, respectively. Similarly, in headline dataset the four
models report 0.722, 0.720, 0.721 and
0.760 cosine similarities, respectively.
We analyze the predictions of all the
four individual models (i.e., CNN,
LSTM, GRU & SVR) as reported in
Table  IV and observe that the performances of these systems are numerically
quite similar. However, when we qualitatively analyze the predictions, we observe
the contrasting nature of these individual
models. In most of the case, the predictions of each individual model are nonoverlapping to each other. For some
examples, one system (say A) obtains relatively correct predictions (i.e., prediction

TABLE V Results of the ensemble model for financial sentiment analysis and emotion analysis tasks. Ensemble models (CNN#,
LSTM#, GRU# & SVR#) refer to the best models of CNN, LSTM, GRU & Feature-SVR based models of Table IV.
FINANCIAL SENTIMENT

EMOTION ANALYSIS

ENSEMBLE MODELS

MICROBLOGS

NEWS

ANGER

JOY

SADNESS

FEAR

AVERAGE

E1

CNN4 + LSTM3 + GRU5 + SVR6

0.797

0.765

-

-

-

-

-

E2

CNN1 + LSTM5 + GRU1 + SVR6

0.779

0.786

-

-

-

-

-

E3

CNN1 + LSTM2 + GRU2 + SVR2

-

-

0.747

0.705

0.744

0.769

0.741

E4

CNN2 + LSTM2 + GRU2 + SVR3

-

-

0.731

0.712

0.745

0.772

0.740

E5

CNN2 + LSTM2 + GRU2 + SVR2

-

-

0.738

0.702

0.755

0.768

0.740

E6

CNN2 + LSTM1 + GRU2 + SVR3

-

-

0.732

0.707

0.748

0.779

0.741

TABLE VI Comparison with the state-of-the-art systems. Emotion Analysis: Prayas, IMS & IITP were the ranked first, second & fifth
systems at EmoInt-2017 [7]. Systems [56]-[58] are the recent works evaluated on the EmoInt-2017 datasets. Sentiment Analysis:
ECNU & Fortia-FBK were the top performing systems at SemEval-2017 task 5 [8] for microblogs and news headlines, respectively.
System [59]+ 10-fold CV.
FINANCIAL SENTIMENT

72

EMOTION ANALYSIS

SYSTEMS

MICROBLOGS

NEWS

ANGER

JOY

SADNESS

FEAR

AVERAGE

SYSTEM [59]+

0.726

0.655

-

-

-

-

-

ECNU [60]

0.777

0.710

-

-

-

-

-

FORTIA-FBK [61]

-

0.745

-

-

-

-

-

BASELINE [7]

-

-

0.625

0.635

0.706

0.620

0.647

IITP [62]

-

-

0.649

0.657

0.709

0.713

0.682

SYSTEM [56]

-

-

0.723

0.671

0.735

0.725

0.713

SYSTEM [57]

-

-

0.716

0.692

0.733

0.728

0.717

IMS [63]

-

-

0.705

0.690

0.767

0.726

0.722

SYSTEM [58]

-

-

0.718

0.717

0.771

0.729

0.734

PRAYAS [64]

-

-

0.732

0.732

0.765

0.762

0.747

PROPOSED SYSTEM

0.797

0.786

0.747

0.712

0.755

0.779

0.748

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2020
IEEE Computational Intelligence Magazine - February 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2020
