Computational Intelligence - August 2015 - 39

C. Compared Approaches

Table 3 Performance comparison for different datasets on running time (s).

As stated above, Liner SVM is adopted as the
DaTaseT
PCa
sParse PCa
rP
rP+PCa
msDa
srP+PCa
ssrP
standard classifier. Since the datasets here are
very sparse, the sparsity problem of the procomp vs rec
453
5012
9
24
75
31
27
jected data in SRP space may still exist. Therecomp vs sci
442
5031
10
27
95
32
27
fore, PCA is applied on the projected data in
comp vs talk
398
4588
8
20
177
28
24
SRP to obtain a non-sparse data. Since PCA
rec vs sci
458
4942
12
26
183
32
27
is conducted on a low-dimensional space, the
rec vs talk
395
4226
9
22
70
28
24
extra computation induced is negligible. The
sci vs talk
395
4226
8
23
142
29
26
following methods are compared: (1) PCA:
Farm ads.
594
3884
40
137
3884
97
78
Data is processed by PCA. We project data
onto a low-dimensional space with PCA.
Then, Linear SVM is applied. (2) Sparse PCA3: Sparse PCA is
will be converged into 100 when m = 10 in Eq. (12). And for
applied to the datasets. After projected by the sparse transforfair comparison, the nonzero entries in each column of projecmation matrix, data is classified by the linear SVM classifier. (3)
tion matrix in Sparse PCA is set to the number of subsampled
RP: Random Projection is adopted to map data into a lowfeatures in our proposed approaches. For mSDA4, we adopt the
dimensional space. Then, we use Linear SVM to classify the
high-dimensional extension of this algorithm, which is more
projected data. (4) RP followed by PCA: PCA is performed on
efficient for high-dimensional data. The hyperparameters for
the latent space found by RP to reduce the sparsity of the data,
mSDA are chosen as follows: the noise intensity is set to 0.5
and then, Linear SVM is applied to the low-dimensional space
and the number of features in each partition for high-dimenderived by RP plus PCA. This setting is designed for fair comsional settings is 3,000. In SDA, the final representation is a
parison with our proposed SRP followed by PCA. (5) mSDA:
concatenation of the original input and the output of each
marginalized stacked denoising autoencoder [21] realizes an
layer. Since the original dimension is very high, we set the
efficient training of Stacked denoising autoencoder. The robust
number of layer to 1 to avoid an extreme high-dimensional
representation learned by mSDA is fed into Linear SVM for
vector and hence very high training cost for the subsequent
classification. (6) SRP followed by PCA: SRP is used to learn a
classifier learning process.
discriminative latent space and PCA is used subsequently to
reduce the sparsity of data. At last, Linear SVM is performed to
D. Performance Comparison on All Datasets
classify data with the new representation. (7) SSRP: Stacked
The experimental results on these seven datasets are shown in
SRP is trained. Then, the output of the last layer forms the
Table 3 and Table 4. Since random assignments are adopted in
latent representation. The Linear SVM is applied on the transRP, RP followed by PCA and our proposed approaches
formed data.
including SRP and SSRP, ten trials are conducted for these
Here, some user-specified parameters in these approaches
four methods on each dataset. The average classification accuraare defined as follows: In SRP, the number of features selected
cy and its corresponding standard variance are taken. In addiduring each random feature sampling d s is set to 6 d @, where
tion, the average running time is also given. As can be seen, our
proposed approaches including SRP and SSRP achieve better
d denotes the original dimension. SRP is adopted here to
classification performance in all of these experiments than
reduce the dimension of data to 10 6 d @ firstly. Then, PCA is
other random approaches. The performance of SRP is only a
applied to reduce the dimension of data to 100 further. The
bit inferior to that of PCA. Except for rec vs talk dataset, SSRP
regularization parameters h in SRP and each layer of SSRP are
determined with 5-fold crossvalidation on the training data.
Table 4 Performance comparison for different datasets on classification accuracy (%). Bold face
Here, the regularization parameindicates best performance.
ters in different layers of SSRP
are unified as a single value. The
DaTaseT
PCa
sParse PCa
rP
rP+PCa
msDa
srP+PCa
ssrP
dimension relationship paramecomp vs rec
93.2
88.3
72.1 ! 2.1
85.7 ! 1.3
94.0
91.6 ! 0.06
94.4 ! 0.13
ter in SSRP m is also unified to
comp vs sci
86.8
81.8
67.6 ! 3.0
80.5 ! 1.7
88.9
84.5 ! 0.08
89.1 ! 0.11
10. Another hyperparameter in
94.9
comp vs talk
94.0
89.7
74.5 ! 2.5
88.6 ! 1.2
92.5 ! 0.11
94.9 ! 0.09
SSRP: the number of layers L is
rec vs sci
88.9
82.2
64.5 ! 2.3
79.3 ! 0.9
87.0
88.9 ! 0.09
90.4 ! 0.13
set to 20. For other dimension90.7
rec vs talk
84.6
66.7 ! 3.8
82.1 ! 1.9
89.7
89.4 ! 0.08
90.6 ! 0.10
ality reduction methods, the
sci vs talk
86.7
82.5
66.0 ! 3.1
80.0 ! 2.3
87.2
81.2 ! 0.12
88.7 ! 0.15
dimension of the latent space r
Farm ads.
86.0
86.3
75.4 ! 2.7
82.7 ! 1.1
84.3
86.7 ! 0.10
88.8 ! 0.19
is set to 100, considering the
dimension reduced by SSRP
91.0
average
89.4
85.0
69.5
82.7
89.4
87.8
3
The code for sparse PCA has been kindly provided at http://www2.imm.dtu.dk/
projects/spasm/.

The code for mSDA has been kindly provided at http://www.cse.wustl.edu/~mchen.

august 2015 | IEEE ComputatIonal IntEllIgEnCE magazInE

http://www2.imm.dtu.dk/ http://www.cse.wustl.edu/~mchen

Table of Contents for the Digital Edition of Computational Intelligence - August 2015