Computational Intelligence - November 2017 - 53

source task, where either the domains or the tasks differ [65]. One
suitable use case would be an optimization task of an aircraft
engine in which source information would originate from a
design effort to optimize a different engine configuration comprising several recurring parameters and outputs. Another potential use case would be to investigate the addition of new design
parameters or an altogether different set of design parameters relative to a prior (source) task.
B. Multi-Task Learning

multi-task regression model built from data originating from various related design efforts can be used to boost multiple optimization algorithms at the same time-in the spirit of multi-task
optimization [54], [69]-[71]. Approaching the problem in this
manner facilitates knowledge transfer and reduces the overall
computational cost of the design effort as the same regions of the
parameter space need not be searched repeatedly [54], thereby
resulting in the need for fewer expensive numerical (finite element
or finite volume) simulations. It is worth noting here that, although
the dimensionality for each of the subproblems is expected to be
significantly lower than for the entire design space, it could be the
case that the common set of input parameters defines a large space
for the multi-task regression. In such scenarios, while the arguments in favor of savings in simulations would still hold, the process of multitasking does not rule out the application of classic
approaches, such as space reduction techniques [72] (the beginning of Section IV elaborates more on this point).

In real-world aircraft designs, it is not uncommon for the total
number of design parameters throughout the entire system to
number in the thousands. Consider, for example, a generic turbofan engine used by most commercial airliners. Its functionality
depends on a host of interdependent systems such as the compressors, turbine, and combustion chamber-each of which is
itself reliant on thousands of design variables. Due to effects such
as the curse of dimensionality [66] coupled with the high computational cost of precision simulations, a monolithic optimizaC. Multi-View Learning
tion of the entire design space would be prohibitively expensive.
Due to the computationally expensive nature of disciplinary analAs such, a common practice is to analyze only a small subset
yses, a common strategy is to approximate the numerical simulaof these design parameters (which constitutes a subproblem).
tions using models of varying fidelity. In certain situations, physical
Consequently, there are often different variations of these subphenomena or structures of interest can be analyzed using either a
problems that are conditioned on the specific choice of all
full 3-D view or even a restricted 2-D/1-D view, depending on
remaining design parameters. A typical example in aircraft design
the types of statistics the engineer is looking for. For example,
of such a situation would be the optimization of an aircraft
while analyzing plate-like structures (which are common in airwing. In each variation, the design team may aim to optimize
craft), one may consider a 3-D discretization if through-thickness
performance parameters (e.g. lift, drag, and weight) as a function
effects are of interest, or may emphasize on the 2-D discretization
of the geometry of the wing [67], conditioned on certain fixed
if only the in-plane effects are of paramount importance [73].
variables (e.g. materials, and flight conditions) subsequently varThus, data that provides an understanding of the mechanics from
ied as part of distinct studies. To elaborate, one variation might
different perspectives is made available in situations where comstudy a wing utilizing composite material A while another,
putational budget is a prime limitation.
composite material B; each with its own
unique set of material properties.
The domain of multi-task learning
is uniquely suited for these types of
Project A
Project B
problems. In contrast to transfer learning which leverages on past experiences, multi-task learning assumes that
there are multiple related tasks that can
be simultaneously solved in order to
Initialization
Initialization
facilitate knowledge transfer [53], [54],
Find Optimal
Find Optimal
Solutions
Solutions
[68]. In the context of multi-task regression, the model aims to learn a mapping as follows: f : X " R N where X
Multitask
represents a common set of input paramSurrogate
eters and the N outputs are considered
Sample New
Sample New
Modeling
to belong to different tasks [53].
Candidate
Candidate
By learning related tasks together
Solution
Solution
using a shared representation, the generGeneric Surrogate-Assisted
Generic Surrogate-Assisted
alization performance is contended to
Optimization Algorithm
Optimization Algorithm
improve by exploiting the shared knowledge in the training signals of the related
FIGURE 2 Illustration of multi-task learning applied to generic optimization algorithms for two
task(s). As illustrated in Figure 2, from related projects. Both Projects A and B are assisted by a multi-task surrogate model constructed
an aircraft design perspective, a single from a combination of data from both projects.

NOVEMBER 2017 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

Table of Contents for the Digital Edition of Computational Intelligence - November 2017