IEEE Computational Intelligence Magazine - February 2021 - 92

is selected as the minority class (17
instances). To ensure the same class
imbalance ratio in both training and test
sets, stratified sampling is employed to
split a dataset into a training set and a test
set: 70% as the training set and 30% as
the test set.
B. Baseline Methods

Sampling methods are popular in solving
the issue of class imbalance. Therefore, we
compare ICS-GP with GP methods where
sampling methods are used to address the
issue of class imbalance. SMOTE [45] and
adaptive synthetic sampling approach
(ADASYN) [22]) are commonly used
oversampling methods, which are employed
to re-balance an unbalanced training set.
Then, the overall classification accuracy
^ Acc = ^TP + TN h/^TP + FN + TN +
FP hh is used as a fitness function by
GP. The two methods are denoted as
GPSMOT E and GPADASYN , respectively.
GP can use a fitness function to
directly avoid developing biased classifiers. Accordingly, ICS-GP is compared
with GP methods using different fitness functions, including the weighted-average classification accuracy
^ Ave = W # ^TP/ ^TP + FN hh + (1 - W )
# ^TN/ ^TN + FP hh, where W = 0.5)
G_Mean , the average mean squared
error (denoted as Amse) [1], Corr [1],
Dist [1], and Wilcoxon-Mann-Whitney
WMW (Aucw) [1], [46]. GP with each
of these fitness functions is denoted as
GPAve , GPG_Mean , GPAmse, GPCorr , GPDist
and GPAucw, respectively.
In unbalanced classification, Ave and
G_Mean are commonly used fitness
functions. GPAve and GPG_Mean use the
standard classification strategy (i.e.,
threshold TH = 0 is used to separate
the original output values of a program
to predict the majority class and the
minority class). As acknowledged, AUC
is an important metric in unbalanced
classification, but GP using it as a fitness function is much more time-consuming than GP using accuracy
measures as a fitness function. Corr and
Dist have been proposed as the AUC
approximation metrics to be used by
GP as a fitness function in unbalanced
classification. Auc w provides a direct

92

estimator for the AUC metric. Aucw is
defined as

/ /

	

Auc w =

I wmw (Pi, P j)

i ! Min j ! Maj

#Min # #Maj

(14)

where
I wmw ^Pi, P j h = )

1, Pi 2 P j and Pi $ 0
,
0, otherwise

Pi and Pj are two output values of a program P taking instance i from Min and
instance j from Maj as inputs.
In [40], [41], GP with cost-sensitive
learning has been investigated. We compare ICS-GP with a cost-sensitive GP
method (denoted as GPRCw ) [40], GP
with a cost-based fitness function (de--
noted as GPCF) [41], and GP with a
boundary based classification strategy
(denoted as GPBC) [41].
Apart from GP methods, ICS-GP is
also compared with classification algorithms from machine learning, where
sampling methods (i.e., SMOTE and
ADASYN) are used to solve the issue
of class imbalance. The classification
algorithms include 1-nearest neighbours (1NN), decision tree (DT), random forests (RF), gradient boosting
decision tree (GBDT), naive bayes
(NB) and multilayer perceptron (MLP).
KNN is a lazy learning-based classification method, where the training
instances are used as prototypes of the
classifiers. DT uses a tree-like representation, which is easier to interpret. RF
and GBDT are variants of DT, based
on ensemble learning. Bayesian classifiers are a kind of probabilistic-based
classification algorithm, in which NB
classifiers are the most common one.
MLP is a feed-forward artificial neural
network. 1NN and NB are run once
because they are deterministic, while
DT, RF, GBDT and MLP are conducted 30 times.
C. Parameter Settings

For GP methods, the population size is
1024 for 50 generations. Population
sizes of 512 and 1024 are the common
settings for GP methods [47]. Because of
the complexity of classification problems
to be solved (i.e., with larger number of

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2021

features), we chose 1024 as a population
size to search for optimal solutions. The
initial population is generated by ramped
half-and-half, which is the most commonly used initialization method in GP
[9]. For each generation, good individuals are selected by tournament selection
[9]. Tournament selection amplifies small
differences of individuals in fitness to
select the better individuals. In this paper,
tournament size k is set to 6. To generate
a new population (with 1024 individuals), 1023 individuals are generated by
standard sub-tree crossover and sub-tree
mutation [9], and 1 individual is directly
inherited from the previous generation
(i.e., elitism). The probabilities of crossover and mutation are set to 0.8 and 0.2,
respectively, which are also very common settings for GP methods [19]. Note
that the proposed method is based on
strongly-typed genetic programming [9].
Therefore, the type constraints are used
to ensure the valid offspring produced by
crossover and mutation. The maximum
tree depth is limited to 10. After 50 generations, the evolutionary learning process of a GP method is terminated (i.e.,
a termination criterion).
For the proposed method, the function set and the terminal set are reported in Table 1. The baseline GP methods
are based on standard tree-based GP,
rather than STGP. For baseline GP
methods, their function set includes +,
-, ×, % (protected division) and If function; their terminal set includes all features and a random constant. Each GP
method had been conducted for 30 runs
independently with 30 different random
seeds (All GP methods use the same set
of random seeds).
VI. Results and Discussions

AUC is the most widely used metric in
unbalanced classification because it is
invariant to uneven class distributions
[1]. The AUC results of the proposed
method (i.e., ICS-GP) and baseline GP
methods are reported in Table  3, and
the results of non-GP methods are
reported in Table  4. Furthermore, the
Wilcoxon statistical significance test
[48] is also conducted to compare ICSGP with a baseline method, with a
IEEE Computational Intelligence Magazine - February 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2021
