IEEE Computational Intelligence Magazine - February 2021 - 86

GP has been successfully applied to classification.
GP can not only be used for feature selection and
feature construction, but also can be directly used to
construct classifiers.
When a classification task suffers
from both high dimensionality and class
imbalance issues, it is often very difficult to select good-quality features that
could improve the accuracies of the
majority class as well as the minority
class. If the selected features are biased
towards the majority class, the constructed classifiers using these selected
features are more likely to be biased
towards the majority class [15].
Liu et al. [16] develop a cost-sensitive
principal component analysis for feature extraction, and a variant of particle
swarm optimization (PSO), called chaos
PSO, is proposed for the parameter op--
timization. Zhang et al. [17] propose a
wrapper-based feature selection method,
based on an optimal F_Measure structural
support vector machine. Aydogan et al.
[18] propose a cost-based rough PSO for
classification with high-dimensional
unbalanced data, which simultaneously
performs feature selection and classification.Tran et al. [19] investigate how GP is
used for feature construction and feature
selection in classification with highdimensional data. In [20], feature clustering is used to group the similar features,
and then the best feature is chosen from
each feature cluster to significantly narrow the search space for GP-based feature construction in classification with
high-dimensional data. In [19], [20], GP
uses the balanced accuracy as a fitness
function to equally treat the majority
class and the minority class. However, in
many real-world a-pplications, the
-majority class and the minority class are
not equally important.

feature selection process, a GP tree as a
classifier does not need to use all features. However, learning from unbalanced data, classifiers evolved by GP
may encounter a performance bias.
Sampling methods, like undersampling [21] and oversampling [22], [23],
are used to re-balance unbalanced datasets. Undersampling methods remove
some instances from the majority class,
to enable the number of instances in the
majority class to be the same or roughly
the same as that of the minority class.
Opposite to undersampling methods,
oversampling methods replicate some
instances from the minority class, or create some new instances for the minority
class. In [24], a GP method is proposed,
where the synthetic minority oversampling technique (SMOTE) is used to
solve the problem of class imbalance.
In [1], [25]-[27], new fitness functions
are developed for GP to address the
problem of class imbalance. In [28], a subset of instances are uniformly selected
and used in the fitness evaluation, and an
area under a curve (AUC) approximation
metric is proposed as a fitness function of
GP. Bhowan et al. [29]-[31] systematically
investigate the use of evolutionary multiple-objective optimization to develop
multi-objective GP methods to address
the issue of class imbalance. However,
compared to single-objective methods,
multi-objective methods are usually
time-consuming, due mainly to the
acquisition of the complete Pareto front.
Accordingly, in this paper, we investigate
a new single-objective GP method by
using cost-sensitive learning.

B. Genetic Programming

C. Cost-Sensitive Learning

GP has been successfully applied to classification. GP can not only be used for
feature selection and feature construction, but also can be directly used to
construct classifiers. Due to its built-in

The standard classification algorithms
assume that all instances are equally
important. However, in unbalanced
-classification, this assumption may result
in a performance bias of constructed

86

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2021

classifiers [13]. Moreover, in many realworld applications, different mistakes
cause different losses. For example, in
medical diagnosis, a cancer patient is
mistakenly diagnosed as a healthy person, which is much more troublesome
than that of misdiagnosing a healthy
person as a cancer patient.
Cost-sensitive learning [32] has been
successfully applied to classification tasks
that are potentially associated with different costs. In cost-sensitive learning,
the costs (mainly misclassification costs,
i.e., the penalty of classifying an instance
from one class to another) are considered to treat the different mistakes
differently. The goal of cost-sensitive
learning is to minimize the total cost,
instead of only simply minimizing the
number of mistakes [33]. Note that the
misclassification costs are different from
the total cost. The misclassification costs
are often provided by the domain
experts, and later used by an algorithm
to develop cost-sensitive classifiers. The
total cost is calculated after predictions
of a cost-sensitive classifier on a classification task. A cost matrix is often used to
indicate the misclassification costs. Usually, the misclassification costs have two
types: class-dependent cost (i.e., different
classes have different costs) and instancedependent cost (i.e., every instance has its
own misclassification cost) [34].
Zhang and Zhou [35] develop costsensitive kernel logistic regression and
cost-sensitive k-nearest neighbor, and
apply them to face recognition. In [36],
a cost-sensitive decision tree algorithm is proposed, which incorporates
instance-dependent misclassification
costs into an impurity measure and
pruning criteria. Iranmehr et al. [37]
develop a new cost-sensitive version of
support vector machines (SVMs) with
an extension of a standard loss function
to optimize a classifier with respect to
class-dependent costs and instancedependent costs. Cost-sensitive learning
is frequently applied to binary classification, but it is less investigated for
multi-class classification. Zhou et al.
[34] investigate the use of cost-sensitive
learning for multi-class c- lassification,
and suggest examining the consistency
IEEE Computational Intelligence Magazine - February 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2021
