Computational Intelligence - February 2016 - 47

kernels are used corresponding
performance of the classificaConventional Ensemble Multiple Kernel Learning
to different notions of similarity.
tion system. A fuzzy ensemble
The lear ning algor ithm
on data with missing values
attempts to either find an optiwas addressed in [43], where
C1
D1
K1
mal combination of kernels or
logical neuro-fuzzy systems
pick one from them if using a
was integrated into the AdaC2
D2
K2
specific kernel may be a source
Boost ensemble method. A
of bias. In the second case, we
fuzzy ensemble with high
may use different versions of
dimensional data was addressed
KMKL
CMKL
Cen
input (i.e, different representain [135].
tions possibly from different
In some cases, it is not reasources or modalities).
sonable to classify the samples
Cn-1
Dn- 1
Kn-1
Linear combination methinto a single label or multiple
ods are popular and have two
labels. For example, an experibasic versions: unweighted
enced herb doctor can offer
Cn
Dn
Kn
combination and weighted
several diagnoses with differcombination. Unweighted is
ent confidences according to
similar to bagging which is disthe symptoms. In this case, a
cussed in Section III-A. In the
fuzzy ensemble can also be Figure 2 Comparison of conventional ensemble methods and mulweighted case, different evaluaemployed to fit the ranking of tiple kernel learning. C denotes a classifier, D denotes data and K
denotes a kernel.
tion criteria can be used to
confidences of fuzzy classes
learn the optimal weights.
[136]. The authors showed
In [142], the weights of different kerthat the ranking of output confidences
correspond to using different notions of
nels were associated with the alignment
could fit the ranking of real confidences
similarity or may be using information
of the kernel to a pre-defined 'target' kerwell, and the fitting error would be
coming from multiple sources (different
nel. In their study, kernels with higher
reduced while the number of the weak
representations or different feature subsets).
alignment to the target were proved to
classifiers increases.
These trained ensembles of kernels are
have better generalization ability. The
Fuzzy logic can be combined with
combined using some classification methauthors proposed an eigen decomposimulti-objective optimization to form an
ods, among which the most commonly
tion method to optimize the alignment
ensemble method as well. A multi-classiused is the support vector machine. MKL
of the combined kernel function. Lanckfier coding scheme and an entropy-based
can also be used as base learners of ensemriet et al. [143] directly optimized the
diversity criterion for evolutionary multible methods [140], [141].
alignment of the combined kernel funcobjective optimization algorithms were
The selection of different kernel
tion with semi-definite programming.
proposed in [137] for the design of fuzzy
functions and the optimization of their
However, semi-definite programming has
ensemble classifiers.The use of evolutioncorresponding parameters have been
a very large computational complexity.
ary multi-objective optimization to conextensively studied. MKL is applied in
Hence, it is intractable for a 'big data'
struct an ensemble of fuzzy rule-based
this context by using an ensemble of
problems. In [144], the authors solved this
classifiers with high diversity is described
kernel functions. Numerous research
problem by rewriting it as a semi-infinite
in [138]. In [139], a multi-objective evoworks have demonstrated the advantage
programming problem that can be effilutionary hierarchical algorithm was proof multiple kernels over a pre-defined
ciently solved by recycling the standard
posed to obtain a non-dominated fuzzy
kernel function with optimized parameSVM solver. Based on the notion of
rule classifier set with interpretability and
ter values. This kind of kernel ensemble
alignment, [145] proposed a two-stage
diversity preservation. Moreover, a
is defined as follows:
learning method for MKL. Cortes et al.
reduce-error based ensemble pruning
(4)
k h (x i, x j) = fh (k m (x i, x j) mp = 1)
[146] proposed the notion of local
method was utilized to decrease the size
Rademacher complexity to design new
and enhance the accuracy of the comwhere the key lies in the combination
algorithms for learning kernels which
bined fuzzy rule classifiers.
function fh which can be a linear or
lead to a satisfactory result with a faster
nonlinear function. Here h is used to
convergence rate. Moreover, in Orabona
parameterize the pre-defined kernel
F. Multiple Kernel Learning Based
et al.'s work [147], faster convergence rate
Ensemble Methods
function. Fig. 2 shows the difference
was achieved by stochastic gradient
between the MKL and conventional
In recent years, multiple kernel learning
descent. Xu et al. [148] proposed a frameensemble methods mentioned in Sec(MKL) has become a hot topic in
work of soft margin MKL and demontion III-A.
machine learning and computational
strated that the commonly used loss
Generally speaking, there are two verintelligence [34]. MKL combines an
function could be readily incorporated
sions of MKL. In the first case, different
ensemble of different kernels which may

February 2016 | Ieee ComputatIonal IntellIgenCe magazIne

Table of Contents for the Digital Edition of Computational Intelligence - February 2016