IEEE Computational Intelligence Magazine - February 2021 - 96

TABLE 5 Comparisons with the best AUC (%) results in [56].
SVM
BT

BEST
AUC

ARMSTRONG-2002-V1

FS

GOLUB-1999-V1
LEUKEMIA

1NN

RF

BT

BEST
AUC

99.3

FS

FS

98.6

FS

98.1

SHIPP-2002-V1

FS

GORDON-2002
YEOH-2002-V1

ICS-GP

BT

BEST
AUC

BEST
AUC

96.7

FS

99.9

100

FS

92.7

ROS

95.4

100

MWMOTE

91.5

FS

97.6

98.21

94.6

FS

72.6

FS

80.4

96.3

FS

99.6

FS

99.4

FS

98.6

100

FS

95.0

MWMOTE

95.9

FS

93.8

100

1: BT stands for the best technique.

better than other methods in all cases.
Moreover, Table 5 shows that the FS
technique enables SVM, 1NN and RF
to achieve the best AUC in 15 out of
the 18 cases. This is due mainly to high
dimensionality of these datasets. ICS-GP
has the built-in ability to select informative features when cost-sensitive classifiers are constructed.
D. Comparisons with an Automated
Machine Learning Model

Automated machine learning (AutoML)
has been successfully applied to many
tasks, such as automated model selection
and automated feature engineering [62].
In our experiments, based on AutoML
techniques, an ensemble classifier (denoted as AutoEnsem) is constructed by
developing a pipeline, where the balancing strategy, classifiers with their parameters, and features engineering can be
automatically processed (the implemen-

tation is based on auto-sklearn [63]). The
AUC results of AutoEnsem are reported
in Table 6. Based on Table 6, the best
AUC results achieved by ICS-GP are
the same or better than AutoEnsem.
VII. Analyses of Example
Evolved GP Trees

In this section, we show two trees evolved
by the proposed method on Lung (a
highly-unbalanced dataset, IR  =  8) and
Armstrong-2002-v1 (a slightly-unbalanced dataset, IR = 2), respectively, for
further analysis. The evolved tree on
Lung is shown in Figure 5, and the
evolved tree on Armstrong-2002-v1 is
shown in Figure 6.
Lung has 156 instances, described by
12600 features. In Figure 5, the evolved
tree has 16 nodes in total, where 6 features are selected from 12600 features to
construct a classifier by its left sub-tree. An
evolved cost interval represented by its

right sub-tree is (1.25, 1.50). Based on the
evolved cost interval, two -classification
thresholds are calculated and used by its
left sub-tree (works as a classifier). On the
test set, the AUC score of this tree is
100%, and the accuracies of the majority
class and the minority class are also 100%.
The training time is 3144.32 seconds.
Note that, for the lower and upper bound
values in a cost interval, we here keep two
decimal places, but they are in fact float
values with sixteen decimal places.
Armstrong-2002-v1 has 72 instances
and 1081 features, and its class imbalance
ratio IR is 2. Shown in Figure 6, the
evolved tree has 11 nodes in total. For its
left sub-tree, only one feature (i.e., f 245)
is selected from 1081 features. A cost
interval, i.e., (5.52, 5.54), is represented
by its right sub-tree. The AUC score of
this tree on the test set is 100%, and the
accuracies of the majority class and the
minority class are also 100%. The training time is 131.23 seconds.
By comparing the two trees, the
evolved tree on Lung is a bit more
complicated than that of Armstrong2002-v1. The reason is due mainly to
more features included in Lung. It is
interesting that, for the evolved cost
interval on Lung, its upper bound value
is less than the imbalance ratio IR of
Lung (i.e., 8). However, for the evolved
cost interval on Armstrong-2002-v1, i.e.,
(5.52, 5.54), its lower bound value is
greater than IR of Armstrong-2002-v1
(i.e., 2). Therefore, it is hard to conclude

TABLE 6 Comparisons with AutoEnsem.

DATASET

96

ICS-GP
AutoEnsem BEST
AUC (%)
AUC (%)

ARMSTRONG2002-V1

100

100

GOLUB-1999-V1

100

100

COLON

74.48

89.29

LEUKEMIA

90.18

98.21

SHIP-2002-V1

91.67

96.3

DLBCL

91.67

100

GORDON-2002

100

100

YEOH-2002-V1

96.15

100

TOMLINS2006-V1

100

100

LUNG

98.81

100

Cost-Sensitive Classifier

Classifier

T_Cost

+
f10457

If
×

f10025

MulCost

f4857

(1.36, 1.13)
+

f7944

FIGURE 5 An evolved tree by ICS-GP on Lung.

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2021

f10437
f11348

(1.10, 1.11)
IEEE Computational Intelligence Magazine - February 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2021
