IEEE Computational Intelligence Magazine - February 2021 - 96
TABLE 5 Comparisons with the best AUC (%) results in [56].
SVM
BT
BEST
AUC
ARMSTRONG-2002-V1
FS
GOLUB-1999-V1
LEUKEMIA
1NN
RF
BT
BEST
AUC
99.3
FS
FS
98.6
FS
98.1
SHIPP-2002-V1
FS
GORDON-2002
YEOH-2002-V1
ICS-GP
BT
BEST
AUC
BEST
AUC
96.7
FS
99.9
100
FS
92.7
ROS
95.4
100
MWMOTE
91.5
FS
97.6
98.21
94.6
FS
72.6
FS
80.4
96.3
FS
99.6
FS
99.4
FS
98.6
100
FS
95.0
MWMOTE
95.9
FS
93.8
100
1: BT stands for the best technique.
better than other methods in all cases.
Moreover, Table 5 shows that the FS
technique enables SVM, 1NN and RF
to achieve the best AUC in 15 out of
the 18 cases. This is due mainly to high
dimensionality of these datasets. ICS-GP
has the built-in ability to select informative features when cost-sensitive classifiers are constructed.
D. Comparisons with an Automated
Machine Learning Model
Automated machine learning (AutoML)
has been successfully applied to many
tasks, such as automated model selection
and automated feature engineering [62].
In our experiments, based on AutoML
techniques, an ensemble classifier (denoted as AutoEnsem) is constructed by
developing a pipeline, where the balancing strategy, classifiers with their parameters, and features engineering can be
automatically processed (the implemen-
tation is based on auto-sklearn [63]). The
AUC results of AutoEnsem are reported
in Table 6. Based on Table 6, the best
AUC results achieved by ICS-GP are
the same or better than AutoEnsem.
VII. Analyses of Example
Evolved GP Trees
In this section, we show two trees evolved
by the proposed method on Lung (a
highly-unbalanced dataset, IR = 8) and
Armstrong-2002-v1 (a slightly-unbalanced dataset, IR = 2), respectively, for
further analysis. The evolved tree on
Lung is shown in Figure 5, and the
evolved tree on Armstrong-2002-v1 is
shown in Figure 6.
Lung has 156 instances, described by
12600 features. In Figure 5, the evolved
tree has 16 nodes in total, where 6 features are selected from 12600 features to
construct a classifier by its left sub-tree. An
evolved cost interval represented by its
right sub-tree is (1.25, 1.50). Based on the
evolved cost interval, two -classification
thresholds are calculated and used by its
left sub-tree (works as a classifier). On the
test set, the AUC score of this tree is
100%, and the accuracies of the majority
class and the minority class are also 100%.
The training time is 3144.32 seconds.
Note that, for the lower and upper bound
values in a cost interval, we here keep two
decimal places, but they are in fact float
values with sixteen decimal places.
Armstrong-2002-v1 has 72 instances
and 1081 features, and its class imbalance
ratio IR is 2. Shown in Figure 6, the
evolved tree has 11 nodes in total. For its
left sub-tree, only one feature (i.e., f 245)
is selected from 1081 features. A cost
interval, i.e., (5.52, 5.54), is represented
by its right sub-tree. The AUC score of
this tree on the test set is 100%, and the
accuracies of the majority class and the
minority class are also 100%. The training time is 131.23 seconds.
By comparing the two trees, the
evolved tree on Lung is a bit more
complicated than that of Armstrong2002-v1. The reason is due mainly to
more features included in Lung. It is
interesting that, for the evolved cost
interval on Lung, its upper bound value
is less than the imbalance ratio IR of
Lung (i.e., 8). However, for the evolved
cost interval on Armstrong-2002-v1, i.e.,
(5.52, 5.54), its lower bound value is
greater than IR of Armstrong-2002-v1
(i.e., 2). Therefore, it is hard to conclude
TABLE 6 Comparisons with AutoEnsem.
DATASET
96
ICS-GP
AutoEnsem BEST
AUC (%)
AUC (%)
ARMSTRONG2002-V1
100
100
GOLUB-1999-V1
100
100
COLON
74.48
89.29
LEUKEMIA
90.18
98.21
SHIP-2002-V1
91.67
96.3
DLBCL
91.67
100
GORDON-2002
100
100
YEOH-2002-V1
96.15
100
TOMLINS2006-V1
100
100
LUNG
98.81
100
Cost-Sensitive Classifier
Classifier
T_Cost
+
f10457
If
×
f10025
MulCost
f4857
(1.36, 1.13)
+
f7944
FIGURE 5 An evolved tree by ICS-GP on Lung.
IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2021
f10437
f11348
(1.10, 1.11)
IEEE Computational Intelligence Magazine - February 2021
Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2021
IEEE Computational Intelligence Magazine - February 2021 - Cover1
IEEE Computational Intelligence Magazine - February 2021 - Cover2
IEEE Computational Intelligence Magazine - February 2021 - 1
IEEE Computational Intelligence Magazine - February 2021 - 2
IEEE Computational Intelligence Magazine - February 2021 - 3
IEEE Computational Intelligence Magazine - February 2021 - 4
IEEE Computational Intelligence Magazine - February 2021 - 5
IEEE Computational Intelligence Magazine - February 2021 - 6
IEEE Computational Intelligence Magazine - February 2021 - 7
IEEE Computational Intelligence Magazine - February 2021 - 8
IEEE Computational Intelligence Magazine - February 2021 - 9
IEEE Computational Intelligence Magazine - February 2021 - 10
IEEE Computational Intelligence Magazine - February 2021 - 11
IEEE Computational Intelligence Magazine - February 2021 - 12
IEEE Computational Intelligence Magazine - February 2021 - 13
IEEE Computational Intelligence Magazine - February 2021 - 14
IEEE Computational Intelligence Magazine - February 2021 - 15
IEEE Computational Intelligence Magazine - February 2021 - 16
IEEE Computational Intelligence Magazine - February 2021 - 17
IEEE Computational Intelligence Magazine - February 2021 - 18
IEEE Computational Intelligence Magazine - February 2021 - 19
IEEE Computational Intelligence Magazine - February 2021 - 20
IEEE Computational Intelligence Magazine - February 2021 - 21
IEEE Computational Intelligence Magazine - February 2021 - 22
IEEE Computational Intelligence Magazine - February 2021 - 23
IEEE Computational Intelligence Magazine - February 2021 - 24
IEEE Computational Intelligence Magazine - February 2021 - 25
IEEE Computational Intelligence Magazine - February 2021 - 26
IEEE Computational Intelligence Magazine - February 2021 - 27
IEEE Computational Intelligence Magazine - February 2021 - 28
IEEE Computational Intelligence Magazine - February 2021 - 29
IEEE Computational Intelligence Magazine - February 2021 - 30
IEEE Computational Intelligence Magazine - February 2021 - 31
IEEE Computational Intelligence Magazine - February 2021 - 32
IEEE Computational Intelligence Magazine - February 2021 - 33
IEEE Computational Intelligence Magazine - February 2021 - 34
IEEE Computational Intelligence Magazine - February 2021 - 35
IEEE Computational Intelligence Magazine - February 2021 - 36
IEEE Computational Intelligence Magazine - February 2021 - 37
IEEE Computational Intelligence Magazine - February 2021 - 38
IEEE Computational Intelligence Magazine - February 2021 - 39
IEEE Computational Intelligence Magazine - February 2021 - 40
IEEE Computational Intelligence Magazine - February 2021 - 41
IEEE Computational Intelligence Magazine - February 2021 - 42
IEEE Computational Intelligence Magazine - February 2021 - 43
IEEE Computational Intelligence Magazine - February 2021 - 44
IEEE Computational Intelligence Magazine - February 2021 - 45
IEEE Computational Intelligence Magazine - February 2021 - 46
IEEE Computational Intelligence Magazine - February 2021 - 47
IEEE Computational Intelligence Magazine - February 2021 - 48
IEEE Computational Intelligence Magazine - February 2021 - 49
IEEE Computational Intelligence Magazine - February 2021 - 50
IEEE Computational Intelligence Magazine - February 2021 - 51
IEEE Computational Intelligence Magazine - February 2021 - 52
IEEE Computational Intelligence Magazine - February 2021 - 53
IEEE Computational Intelligence Magazine - February 2021 - 54
IEEE Computational Intelligence Magazine - February 2021 - 55
IEEE Computational Intelligence Magazine - February 2021 - 56
IEEE Computational Intelligence Magazine - February 2021 - 57
IEEE Computational Intelligence Magazine - February 2021 - 58
IEEE Computational Intelligence Magazine - February 2021 - 59
IEEE Computational Intelligence Magazine - February 2021 - 60
IEEE Computational Intelligence Magazine - February 2021 - 61
IEEE Computational Intelligence Magazine - February 2021 - 62
IEEE Computational Intelligence Magazine - February 2021 - 63
IEEE Computational Intelligence Magazine - February 2021 - 64
IEEE Computational Intelligence Magazine - February 2021 - 65
IEEE Computational Intelligence Magazine - February 2021 - 66
IEEE Computational Intelligence Magazine - February 2021 - 67
IEEE Computational Intelligence Magazine - February 2021 - 68
IEEE Computational Intelligence Magazine - February 2021 - 69
IEEE Computational Intelligence Magazine - February 2021 - 70
IEEE Computational Intelligence Magazine - February 2021 - 71
IEEE Computational Intelligence Magazine - February 2021 - 72
IEEE Computational Intelligence Magazine - February 2021 - 73
IEEE Computational Intelligence Magazine - February 2021 - 74
IEEE Computational Intelligence Magazine - February 2021 - 75
IEEE Computational Intelligence Magazine - February 2021 - 76
IEEE Computational Intelligence Magazine - February 2021 - 77
IEEE Computational Intelligence Magazine - February 2021 - 78
IEEE Computational Intelligence Magazine - February 2021 - 79
IEEE Computational Intelligence Magazine - February 2021 - 80
IEEE Computational Intelligence Magazine - February 2021 - 81
IEEE Computational Intelligence Magazine - February 2021 - 82
IEEE Computational Intelligence Magazine - February 2021 - 83
IEEE Computational Intelligence Magazine - February 2021 - 84
IEEE Computational Intelligence Magazine - February 2021 - 85
IEEE Computational Intelligence Magazine - February 2021 - 86
IEEE Computational Intelligence Magazine - February 2021 - 87
IEEE Computational Intelligence Magazine - February 2021 - 88
IEEE Computational Intelligence Magazine - February 2021 - 89
IEEE Computational Intelligence Magazine - February 2021 - 90
IEEE Computational Intelligence Magazine - February 2021 - 91
IEEE Computational Intelligence Magazine - February 2021 - 92
IEEE Computational Intelligence Magazine - February 2021 - 93
IEEE Computational Intelligence Magazine - February 2021 - 94
IEEE Computational Intelligence Magazine - February 2021 - 95
IEEE Computational Intelligence Magazine - February 2021 - 96
IEEE Computational Intelligence Magazine - February 2021 - 97
IEEE Computational Intelligence Magazine - February 2021 - 98
IEEE Computational Intelligence Magazine - February 2021 - 99
IEEE Computational Intelligence Magazine - February 2021 - 100
IEEE Computational Intelligence Magazine - February 2021 - 101
IEEE Computational Intelligence Magazine - February 2021 - 102
IEEE Computational Intelligence Magazine - February 2021 - 103
IEEE Computational Intelligence Magazine - February 2021 - 104
IEEE Computational Intelligence Magazine - February 2021 - 105
IEEE Computational Intelligence Magazine - February 2021 - 106
IEEE Computational Intelligence Magazine - February 2021 - 107
IEEE Computational Intelligence Magazine - February 2021 - 108
IEEE Computational Intelligence Magazine - February 2021 - Cover3
IEEE Computational Intelligence Magazine - February 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com