Computational Intelligence - November 2015 - 33

SenticNet. However, no lexical resource can be
The common-sense knowledge features were the
absolutely complete, and in any text there will
be some words or concepts still absent in the
main component of our feature vectors. They were
current version of the knowledge base. Instead
extracted from the AffectiveSpace lexical resource.
of simply ignoring them, we resort to a backup processing method based on statistical
machine learning. Namely, for the concepts that are absent in
Common-sense knowledge features were the main comSenticNet, we extract features as described below and use a CI
ponent of the feature vectors. They were extracted from the
classifier trained on three well-known sentiment analysis datasets.
AffectiveSpace [33] for those concepts that were represented
in the multidimensional vector space of commonsense
knowledge. The latter resource assigns to each concept 100
A. Datasets Used
real-number values. The common-sense knowledge feature
We used the following three datasets to train the classifier, as
vector for the whole sentence was obtained by coordinatewell as to test the proposed method:
wise summation of individual 100-dimensional vectors for
1) Movie Review Dataset: This dataset contains one thousand
each concept present in the sentence and found in Affecpositive reviews and one thousand negative reviews extracttiveSpace.
ed from the benchmark corpus of movie reviews created by
The sentic feature directly represented the polarity of each
Pang and Lee [35]. The corpus was collected from the webconcept extracted from SenticNet; for the complete sentence
site rottentomatos.com, where the texts are written by
the polarities of individual concepts were summed up. This
experts. The corpus has been pre-processed. The reviews
represents the baseline "old way" shown in Figure 1(b).
were labeled by Pang and Lee at the document level, and
Finally, the following three features reflected the formal
later by Socher et al. [7], who labeled each individual sencharacteristics of the sentence. The part-of-speech feature was a
tence. The sentences are labeled with five labels, from strong
3-dimensional vector reflecting the number of adjectives in the
or weak positive to neutral to weak or strong negative. For
sentence, the number of adverbs, and the number of nouns.
the experiments, however, we did not use neutral sentences,
The binary negation feature indicated whether there was any
and for all others, we only considered binary polarity. The
negation in the sentence. In a similar way, the binary modificaresulting corpus we used in the experiments includes 4,800
tion feature reflected whether in the dependency structure of
positive sentences and 4,813 negative sentences.
the sentence there existed a word modified by a noun, an
2) Blitzer Dataset: Another dataset we used in the experiments
adjective, or an adverb. The latter feature, however, was not
was developed by Blitzer et al. [10]. It includes one thousand
found to be useful in the experiments.
positive documents and one thousand negative documents
for each one of the seven domains, of which we only used
the electronics domain. From this corpus, we selected at ranC. Classification
dom 3,500 sentences marked other than neutral belonging
We selected 60% of the sentences from each of the three
to positive reviews, and the same amount of non-neutral
datasets as the training set for the classification. The sensentences belonging to negative reviews. Then we manually
tences from each dataset were randomly drawn in such a
labeled them with polarity, since we found that the polarity
way to balance the dataset with 50% negative sentences and
of an individual sentence not always corresponded to the
50% positive sentences. We used a novel CI technique called
polarity of the whole document, since in many cases the
extreme learning machine (ELM) [36], [37], which is a
reviews were balanced and contained both positive and negrecently developed type of single-hidden layer feedforward
ative considerations. This procedure resulted in 3,800 posinetworks, with hidden layer that does not require tuning.
tive sentences and 3,410 negative sentences.
ELM was found to outperform state-of-the-art methods
3) Amazon product review dataset: We crawled the reviews of
such as SVM, in terms of both accuracy as well as training
453 mobile phones from http://amazon.com. Each review
time. In the experiments, we obtained an overall 71.32%
was split into sentences, and each sentence was then manuaccuracy on the final dataset described in Table 1 using
ally labelled by its sentiment labels. Finally, we obtained
ELM and 68.35% accuracy using SVM.
115,758 sentences, out of which 48,680 were negative,
2,957 sentences neutral and 64,121 positive. In this experiment, we only employed positive and negative sentences.
Table 1 Datasets to train and test CI classifiers.
So, the final Amazon dataset contained 112,801 sentences
annotated as either positive or negative.
Number of TraiN- Number of
DaTaSeT

iNg SeNTeNceS

TeST SeNTeNceS

B. Feature Set

Movie Review dataset

5,678

3,935

As classification features, we used common-sense knowledge
features, a sentic feature, a part-of-speech feature, a negation
feature, and a modification feature.

BlitzeR-deRived dataset

4,326

2,884

aMazon dataset

67,681

45,120

Final dataset

77,685

51,939

November 2015 | Ieee ComputatIoNal INtellIgeNCe magazINe

http://www.rottentomatos.com http://www.amazon.com

Table of Contents for the Digital Edition of Computational Intelligence - November 2015