IEEE Computational Intelligence Magazine - February 2020 - 69

which allows word embeddings to be
fine-tuned during the training.

TABLE II Dataset statistics.

V. Experiments, Results and
Analysis

DATASETS

DOMAIN

TRAIN

DEV

TEST

EMOTION ANALYSIS (WASSA-2017) [7]

ANGER

857

760

FEAR

1,147

110

995

JOY

823

714

SADNESS

786

673

MICROBLOGS

1,700

800

NEWS

1,142

491

A. Dataset

We evaluate our model on the datasets
of eighth Workshop on Computational
Approaches to Subjectivity, Sentiment and
Social Media Analysis shared task on emotion intensity (EmoInt-2017) [7] for emotion analysis.The datasets of SemEval-2017
shared task on 'Fine-Grained Sentiment
Analysis on Financial Microblogs and
News' [8], are used for sentiment analysis.
The EmoInt-2017 datasets [7] contain
generic tweets representing four emotions, i.e., joy, fear, anger and sadness.
Datasets of SemEval-2017 [8] comprise of financial texts from microblogs
(Twitter and StockTwits) and news
(Yahoo finance). For the experimental
purpose, we perform five-fold cross validation for model tuning and hyperparameter selection. According to the respective
description papers, SemEval-2017 dataset
[8] was manually annotated by three
human financial experts while Emotion
Intensity [7] dataset was created using the
Best-Worst Scaling technique [52]. Detailed statistics of both the datasets are
presented in Table II.
B. Preprocessing

We use NLTK [53] for tokenization.
Since the contents were derived from
the Internet, pre-processing is of paramount importance due to lack of proper
grammar and structures. Since URLs,
user names and numbers usually do not
carry any polar sentiments, we replace
these with the tags: , and
, respectively. For example, we
replace 'www.twitter.com' by and '@
JonSnow' by . After stripping off
excess white spaces, all the characters
were converted to lowercase. Additionally, all HTML entities were converted to
their corresponding unicode characters
such as '&' was replaced by 'and'.
Hashtags carry meaningful information
and are relevant to extract underlying
emotions and sentiments. We first stripoff # symbol from the hashtags and then

SENTIMENT ANALYSIS (SEMEVAL-2017) [8]

split the resulting token into constituent
words. For example, '#GreatDayEver' is
converted to 'Great Day Ever'. We
employ python-based WordSegment 1
module for the segmentation of hashtags.
Finally, we perform normalization of
noisy text by employing the following
set of heuristics in line with [54].
❏ Elongation of a valid word: To convey their state of emotions or sentiments, users tend to express through
elongation of a valid word, e.g., 'joooyy',
'gooood', etc. We define a heuristics that
identifies all such elongated words and
process them into valid dictionary
words by iteratively dropping the consecutive sequence of characters. For
example 'joooyy' and 'gooood' are converted to 'joy' and 'good', respectively.
❏ Frequent noisy term: Due to the
character limit in Twitter, usage of
abbreviations and slang terms in
tweets are in common practice among
users, e.g., 'grt', 'g8' for 'great'. To
handle such cases we created a dictionary of commonly used abbreviations and slang terms along with
their expanded valid forms. We then
perform a lookup in the dictionary
for each token in a tweet and on the
success we use its expanded valid
form for further processing. We compile the dictionary of the frequent
noisy term by consulting the datasets
of WNUT-2015 shared task on Twitter Lexical Normalization [55].
❏ Verb present participle: By careful
inspection of tweets, it is observed
that users have a common practice to
skip the characters 'g' or 'i' from the
present participle form of a verb, i.e.,
'ing' form of the verb. For example,
1

https://github.com/grantjenks/wordsegment

'enjoying' is written as 'enjoyin' or
'enjoyng'. We correct these cases by
applying a heuristics that considers
all the verbs that end with either 'in'
or 'ng' and convert them into valid
present participle form of the verb.
❏ Expand contraction: Twitter users
generally tend to merge two words by
introducing an apostrophe (') symbol
in place of few in between character
sequences, e.g., the contraction "i've"
belongs to valid words "i have". Such
practice saves a few crucial characters
in a tweet and can be utilized for extra
words. We create a list of such contractions and their expanded forms by
consulting the WNUT-2015 datasets
[55].We apply a heuristic that identifies
contracted tokens in a tweet and converts them to its normalized form.
C. Experiments

For evaluation of the proposed models,
we employ the Pearson correlation
coefficient and cosine similarity score
for the problems of emotion intensity
and sentiment score, respectively. The
choice of evaluation metrics was derived
from the guidelines of the shared tasks
on EmoInt-2017 [7] and SemEval-2017
[8]. Pearson correlation coefficient measures the linear correlation between the
actual and predicted scores, whereas the
cosine similarity score measures the
degree of agreement between the actual
and predicted values.
We separately train and tune all the
deep learning systems (CNN, LSTM
and GRU) over different word embeddings-pretrained, financial and the
DAWE. As mentioned earlier, we do not
employ financial word embedding for
the generic emotion analysis task. The
network architecture (dimensions, layers,

FEBRUARY 2020 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

http://www.twitter.com

IEEE Computational Intelligence Magazine - February 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2020