IEEE Computational Intelligence Magazine - May 2022 - 53

introduced in [51]. In neural networks for
instance, tasks could be defined by different
loss functions or network topologies, with the
transfer of model parameters between them
leading to better training [52]. More generally,
for scaling-up the evolutionary configuration
of arbitrary ML subsystems, the idea of constructing
auxiliary small data tasks from an otherwise
large dataset was proposed in [53], [54]. The auxiliary
tasks can then be combined with the main task in EMT, accelerating
search by using small data to quickly optimize for the
large dataset; speedups of over 40% were achieved in some cases
of wrapper-based feature selection via an EMT algorithm with
explicit transfer [53]. In another application for feature selection,
the tendency of stagnation of EAs in high-dimensional feature
spaces was lessened by initiating information transfers between
artificially generated low-dimensional tasks [55], [56].
EMT on sets of learning algorithms: Given a training dataset,
EMT could effectively function as an expert ML
practitioner, exploiting knowledge transfers across
non-identical but related domains to speed up
model configuration.
an ensemble (or set) of classification models could be learnt by
simple repetition of classifier evolution. However, this would
multiply computational cost. As an alternative, the study in [58]
proposed a variant of multifactorial genetic programming (MFGP)
for simultaneous evolution of an ensemble of decision trees.
MFGP enabled a set of classifiers to be generated in a single
run, with the transfer and reuse of common subtrees providing
substantial cost savings in comparison to repeated (independent)
runs of genetic programming. Moving upstream in the
data science pipeline, [59] formulated the task of finding optimal
feature subspaces for each base learner in an ensemble as
an MTO problem. An EMT feature selection algorithm was
then proposed to solve this problem, yielding feature subspaces
that often outperformed those obtained by seeking the optimal
for each base learner independently. A similar idea but targeting
the specific case of hyperspectral image classifiers was
presented in [60].
Beyond the training of ML models, the literature evinces
applications of EMT for image processing as well. For sparse
unmixing of hyperspectral images, the approaches in [61], [62]
suggest to first partition an image into a set of homogeneous
regions. Each member of the set is then incorporated as a constitutive
sparse regression task in EMT, allowing implicit genetic
transfers to exploit similar sparsity patterns. Results revealed
faster convergence to near-optimal solutions via EMT, as
opposed to processing pixels or groups of pixels independently.
In [63], a multi-fidelity MTO procedure was incorporated into
the hyperspectral image processing framework. A surrogate
model was used to estimate the gap between low- and highfidelity
evaluations to achieve further improvements in accuracy
and algorithmic efficiency.
EMT across non-identical datasets: It is envisaged that future
cloud-based black-box optimization services shall open up
diverse applications of EMT for automated configuration of
learning algorithms. Comparable services are already on the
horizon, making it possible for users to upload their raw data to
the cloud and have high-quality predictive models delivered
without the need for extensive human inputs [57]. Different
user groups may possess non-identical data, and, as depicted in
Fig. 1, may even pose different device requirements constraining
the transition of trained models from the cloud to the edge.
In such settings, EMT could effectively function as an expert
ML practitioner, exploiting knowledge transfers across nonidentical
but related domains to speed up model configuration.
Early works showing the plausibility of this idea-using a distinct
class of multitask Bayesian optimization algorithms-were
presented in [64], [65].
Recently, an intriguing application of EMT feature selection
to understand the employability of university graduates has
been explored [66]. Students studying different disciplines
(business, engineering, etc.) formed multiple non-identical
cohorts, with the data for each cohort forming a feature selection
task in MTO. Then, by allowing common features/attributes
to be shared through multitasking, efficient identification
of determinants that most influence graduate employment outcomes
was achieved. In [67], a multitask genetic programming
algorithm for feature learning from images was proposed. For a
given pair of related but non-identical datasets, the approach
jointly evolves common trees together with task-specific trees
that extract and share higher-order features for image classification.
The effectiveness of the approach was experimentally verified
for the case of simultaneously solving two tasks, showing
similar or better generalization performance than single-task
genetic programming.
1) Case study in symbolic regression modeling [68]
Many works in the literature have explored multitasking in
connection with genetic programming [70], [71]. Here, a realworld
study of MFGP comprising two symbolic regression
tasks with distinct time series data is considered [68].
The first problem instance contains 260 data points representing
monthly average atmospheric CO2
concentrations collected
at Alert, Northwest Territories, Canada from January 1986
to August 2007. The second problem instance contains 240 data
points representing monthly U.S. No 2 Diesel Retail Prices
(DRP) from September 1997 to August 2017. Two simplified
tasks with reduced time series datasets were also generated by
subsampling of the original data. These were labelled as S_CO2
and S_DRP, respectively. The MFGP was thus applied to solve
three pairs of tasks, i.e., {CO2, S_CO2}, {CO2, DRP} and
{DRP, S_DRP}, each with the goal of deriving a symbolic
(closed-form mathematical) equation mapping elapsed time to
the output prediction. Equations were evolved by minimizing
their root mean square error (RMSE) [68].
MAY 2022 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 53
IEEE Computational Intelligence Magazine - May 2022

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2022
