IEEE Computational Intelligence Magazine - August 2019 - 48

Nevertheless, ComE, and ComE+ can
partially overcome this limitation because they leverage community information, validating the importance of
jointly perform node and community
embedding. Fifth, on average, among all
the baseline, GraRep appears to be the
best performing methods. This suggests
that the higher-order transition probability matrix contains valuable information,
but comes at the price of an overall
higher complexity (O ( ;V ;3 )). Whereas,
on the university Facebook data sets,
M-NMF presents results comparable
with the GraRep models, validating the
importance of modeling communities in
a network structure. Overall, the results
validate the existence of a closed loop
between mesoscopic communities structure and the microscopic nodes structure. That is, community information is
not only useful for community-related
tasks but also node related tasks.
C. Parameter sensitivity

In this section we are going to explore
the performance of the ComE+ algo-

rithm for different parameter setting.
Also, we focus on the comparison
between ComE+ and ComE to evaluate better the variational inference
approach used for community detection
and embedding.
1) Impact of K
As the main difference between ComE+
and ComE is the variational inference
process, we compare the impact of
parameter K for both algorithms. Let us
denote K l as the real number of communities present in datasets. Then, we
generate different embedding with K
equal to the values [3, K l , 10, 20] for
DBLP and [5, 10, K l , 50, 100] for all the
other datasets. This enable us to evaluate the robustness of the proposed
approach w.r.t. the uncertainty in the
number of communities.
From Fig. 3 we make the following
observations. First, on DBLP, neither
ComE nor ComE+ is sensitive to the
setting of K regarding node classification, but both of them are sensitive to
K concerning community detection.

Such behavior indicates that the SVM
can exploit the RBF kernel to separate
the nodes, while the generated node
embedding misleads a traditional clustering algorithm. On the one hand, the
performance of ComE reaches the top
when K = K l , while they reduce by a
relative -140% to -189% in terms of
Conductance and -17.6% to -25.6%
w.r.t. NMI. On the other hand, as
expected, ComE+ is more robust when
K $ K l . That is, the ComE+ performance only varies between -1.4% to
+ 0.2% for the NMI metric and -26.1%
to -2.3% according to Conductance.
Secondly, the models usually perform
the best when K = K l in most cases,
although on rare occasions (e.g., on
Amherst) ComE+ has better performance when K 2 K l. Although the
improvement is not significant, this suggests that the communities present in
the data distribution do not adequately reflect the number of labels. Alternatively, it expresses the tendency of
node embedding methods to create
small sets of homogeneous nodes; thus,

TABLE III Node classification results. Note that all the experiments are conduced with 80% of the total nodes as training set while
the remaining 20% is used for evaluation.
BLOGCATALOG

COME

WIKIPEDIA

MACRO-F1(%)

MICRO-F1(%)

MACRO-F1(%)

MICRO-F1(%)

MACRO-F1(%)

MICRO-F1(%)

26.5 (P = 0.12)

41.7 (P = 0.06)

9.8 (P = 0.15)

44.0 (P = 0.13)

92.2 (P = 0.61)

92.6 (P = 0.47)

COME+

27.1

42.5

10.2

45.4

92.2

92.6

DEEPWALK

22.2 (P < 0.01)

38.3 (P < 0.01)

4.6 (P < 0.01)

28.0 (P < 0.01)

91.2 (P < 0.01)

91.6 (P < 0.01)

LINE

10.9 (P < 0.01)

30.2 (P < 0.01)

5.3 (P < 0.01)

30.5 (P < 0.01)

90.4 (P < 0.01)

91.0 (P < 0.01)

NODE2VEC

24.1 (P < 0.01)

39.9 (P < 0.01)

6.1 (P < 0.01)

31.1 (P < 0.01)

91.5 (P < 0.01)

92.0 (P < 0.01)

GRAREP

23.6 (P < 0.01)

40.9 (P < 0.01)

8.1 (P = 0.09)

33.4 (P < 0.01)

90.6 (P < 0.01)

91.1 (P < 0.01)

M-NMF

15.7 (P < 0.01)

33.8 (P < 0.01)

7.0 (P < 0.01)

33.7 (P < 0.01)

89.6 (P < 0.01)

90.3 (P < 0.01)

PRUNE

4.6 (P < 0.01)

15.6 (P < 0.01)

4.9 (P < 0.01)

35.0 (P < 0.01)

22.4 (P < 0.01)

38.4 (P < 0.01)

ROCHESTER
MACRO-F1(%)

48

DBLP

MICRO-F1(%)

MICH
MACRO-F1(%)

MICRO-F1(%)

AMHERST
MACRO-F1(%)

MICRO-F1(%)

COME

49.7 (P < 0.05)

86.6 (P = 0.42)

36.9 (P = 0.61)

63.2 (P = 0.21)

65.7 (P = 0.5)

91.1 (P = 0.05)

COME+

53.7

86.8

37.3

64.1

66.6

91.6

DEEPWALK

44.1 (P < 0.01)

82.9 (P < 0.01)

33.2 (P < 0.01)

60.9 (P < 0.01)

57.6 (P < 0.01)

88.5 (P < 0.01)

LINE

47.4 (P < 0.01)

85.4 (P < 0.05)

34.1 (P < 0.01)

61.5 (P < 0.01)

59.5 (P < 0.01)

88.9 (P < 0.01)

NODE2VEC

46.6 (P < 0.01)

82.6 (P < 0.01)

34.4 (P < 0.05)

61.6 (P < 0.01)

57.6 (P < 0.01)

89.4 (P < 0.01)

GRAREP

48.8 (P < 0.01)

86.5 (P < 0.01)

35.4 (P = 0.29)

63.0 (P = 0.13)

62.9 (P = 0.07)

91.0 (P = 0.08)

M-NMF

48.3 (P < 0.01)

86.4 (P = 0.23)

34.3 (P < 0.05)

61.6 (P < 0.05)

60.0 (P < 0.05)

90.8 (P < 0.05)

PRUNE

13.2 (P < 0.01)

29.1 (P < 0.01)

11.6 (P < 0.01)

23.6 (P < 0.01)

12.7 (P < 0.01)

27.3 (P < 0.01)

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2019
IEEE Computational Intelligence Magazine - August 2019

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2019
