IEEE Computational Intelligence Magazine - February 2021 - 104

first experiment, results are compatible
with what reported in the literature,
although some differences in perfor-

mance are present due to the use of a
random data split rather than the predefined one used in the original experi-

TABLE IV Mean-squared error on the graph regression tasks.
Results for Homo are in scale of 10-5.
DATASET

AVERAGE

SUM

MAX

GAP

AWSP

1.12 ± 0.03

1.02 ± 0.02

0.90 ± 0.04

1.04 ± 0.05

0.99 ± 0.03

ALPHA

3.15 ± 0.65

2.38 ± 0.64

6.20 ± 0.33

1.89 ± 0.59

31.1 ± 0.37

HOMO

9.24 ± 0.41

9.22 ± 0.51

8.90 ± 0.36

9.04 ± 0.29

8.05 ± 0.29

0.42 ± 0.14

0.50 ± 0.13

110.7 ± 4.5

0.22 ± 0.13

624.0 ± 19.0

ments. The APPNP operator con--sistently
achieves good results on the citation
networks, outperforming the other
methods on Cora and Pubmed, and
coming close to ARMA on CiteSeer.
For graph classification, the results are
sometimes different than what is reported in the literature, due to the standardised architecture that we used in this
experiment. MinCut generally achieves
the best performance followed by DiffPool. We also note that the Flat baseline
often achieves better results than the

Appendix
Experimental Details
This section summarises the architectures and hyperparameters
used in the experiments of Section IV.
A. Node Classification
Hyperparameters:
❏❏ Learning rate: see original papers;
❏❏ Weight decay: see original papers;
❏❏ Epochs: see original papers;
❏❏ Patience: see original papers;
❏❏ Repetitions per method and per dataset: 100;
❏❏ Data: we used Cora, Citeseer and Pubmed. As suggested in
[51], we use random splits with 20 labels per class for training, 30 labels per class for early stopping, all the remaining
labels for testing.
B. Graph Classification
We configure a GNN with the following structure: GCS - Pooling GCS - Pooling - GCS - GlobalSumPooling - Dense, where GCS
indicates a Graph Convolutional Skip layer as described in [26],
Pooling indicates the graph pooling layer being tested, GlobalSumPooling represents a global sum pooling layer, and Dense
represents the fully-connected output layer. GCS layers have 32
units each, ReLU activation, and L2 regularisation applied to
both weight matrices. The same L2 regularisation is applied to
pooling layers when possible. Top-K and SAGPool layers are
configured to output half of the nodes for each input graph. DiffPool and MinCut are configured to output K = Nr /2 nodes at
r /4 nodes at the second layer, where N
r
the first layer, and K = N
is the average order of the graphs in the dataset. When using
DiffPool, we remove the first two GCS layers, because DiffPool
has an internal message-passing layer for the input features. DiffPool and MinCut were trained in batch mode by zero-padding
the adjacency and node attributes matrices. All networks were
trained using Adam with the default parameters of Keras, except
for the learning rate.

104

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2021

Hyperparameters:
❏❏ Batch size: 8;
❏❏ Learning rate: 0.001;
❏❏ Weight decay: 0.00001;
❏❏ Epochs: models trained to convergence;
❏❏ Patience: 50 epochs;
❏❏ Repetitions per method and per dataset: 10;
❏❏ Data: we used the Benchmark Datasets for Graph Kernels as
described in [53], that were modified to contain no isomorphic graphs. For each run, we randomly split the dataset and
use 80% of the data for training, 10% for early stopping, and
10% for testing.
C. Graph Regression
We configure a GNN with the following structure: ECC - ECC GlobalPooling - Dense, where ECC indicates an Edge-Conditioned
Convolutional layer [23] and GlobalPooling indicates the global
pooling layer being tested. ECC layers have 32 units each, and
ReLU activation. No regularisation is applied to the GNN. GAP is
configured to use 32 units. All networks were trained using Adam
with the default parameters of Keras, except for the learning rate.
We use the mean squared error as loss.
Node features are one-hot encodings of the atomic number of
each atom. Edge features are one-hot encodings of the bond
type. The units of measurement for the target variables are: debye
units (D) for Mu, a 30 (a0 is the Bohr radius) for Alpha, and Hartree
(Ha) for Homo and U0 [46].
Hyperparameters:
❏❏ Batch size: 32;
❏❏ Learning rate: 0.0005;
❏❏ Epochs: models trained to convergence;
❏❏ Patience: 10 epochs;
❏❏ Repetitions per method and per dataset: 5;
❏❏ Data: for each run, we randomly split the dataset and use
80% of the molecules for training, 10% for early stopping,
and 10% for testing.

IEEE Computational Intelligence Magazine - February 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2021