Computational Intelligence - November 2014 - 68

known as robust optimal solutions [29].
However, if the changes are large and
continuous, meta-heuristics for tracking
the moving optima will often be developed, which is known as dynamic optimization [42]. Different from the
robustness approach to handling uncertainties, dynamic optimization aims to
track the optimum whenever it
changes. Theoretically this sounds perfect, but practically it is not desired for
two reasons. First, tracking a moving
optimum is computationally intensive,
particularly if the fitness evaluations are
expensive. Second, a change in the
design or solution may be expensive
and frequent changes are not allowed in
many cases. To take these two factors
into account, a new approach to cope
with uncertainties, termed robustness
over time, has been suggested [30]. The
main idea is to reach a realistic trade-off
between finding a robust optimal solution and tracking the moving optimum.
That is, a design or solution will be
changed only if the solution currently
in use is no longer acceptable, and a
new optimal solution that changes
slowly over time, which is not necessarily the best solution in that time instant,
will be sought.
4.2. Big Data in Optimization

Meta-heuristic global optimization of
complex systems cannot be accomplished without data generated in
numerical simulations and physical
experiments. For example, design optimization of a racing car is extremely
challenging since it involves many subsystems such as front wing, rear wing,
chassis and tires. A huge number of
decision variables are involved, which
may seriously degrade the search performance of meta-heuristics. To alleviate
this difficulty, data generated by aerodynamic engineers in their daily work will
be very helpful to determine which
subsystem, or even as a step further
which part of the subsystem, is critical
for enhancing the aerodynamic and
drivability of a car. Analysis and mining
of such data is, however, a challenging
task, because the amount of data is huge,
and the data might be stored in different

forms and polluted with noise. In other
words, these data are fully characterized
by the four V's of big data. In addition, as
fitness evaluations of racing car designs
are highly time-consuming, surrogates
are indispensable in optimization of racing vehicles.
Another example is the computational reconstruction of biological gene
regulatory networks. Reconstruction of
gene regulatory networks can be seen as
a complex optimization problem, where
a large number of parameters and connectivity of the network need to be
determined. While meta-heuristic optimization algorithms have been shown to
be very promising, the gene expression
data for reconstruction is substantially
big data in nature [51]. Data available
from gene expression is increasing at an
exponential rate [59]. The volume of
data is ever increasing with developments in next generation sequence
techniques such as high-throughput
experiments. In addition, data from
experimental biology, such as microarray
data, is noisy, and gene expression experiments rarely have the same growth
conditions and thus produce heterogeneous data sets. Data variety is also significantly increased through the use of
deletion data, where a gene is deleted in
order to determine its regulatory targets.
Perturbation experiments are useful in
reconstruction of gene regulatory networks, which, however, are another
source of variety in biological data. Data
collected from different labs for the
same genes in the same biological network are often different.
It also becomes very important to
develop optimization algorithms that
are able to gain problem-specific
knowledge dur ing optimization.
Acquisition of problem-specific knowledge can help capture the problem
structure to perform more efficient
search. For largescale problems that
have a large number of objectives, such
knowledge can be used to guide the
search through the most promising
search space, and to specify preferences
over the objectives so that the search
will focus on the most important tradeoffs. Unfortunately, sometimes only

IEEE ComputatIonal IntEllIgEnCE magazInE | novEmbEr 2014

limited a-priori knowledge is available
for the problem to be solved. It is
therefore also interesting to discover
knowledge from similar optimization
problems or objectives that have been
previously solved [20]. In this case,
proper re-use of the knowledge can be
very challenging. The relationship
between the challenges in complex systems optimization and the nature of big
data is illustrated in Fig. 2.
4.3. Opportunities and Challenges

As discussed above, big data is widely
seen as essential for the success of the
design optimization of complex systems.
Much effort has been dedicated to the
use of data to enhance the performance
of meta-heuristic optimization algorithms for solving large-scale problems in
the presence of large amounts of uncertainties. It is believed that the boom in
big data research can create new opportunities as well as impose new challenges
to data driven optimization. Answering
the following questions can be central to
converting the challenges posed by big
data into opportunities.
First, how can we seamlessly integrate modern learning and optimization
techniques? Many advanced learning
techniques, such as semi-supervised
learning [63], incremental learning [15],
active learning [47] and deep learning
[10] have been developed over the past
decade. However, these techniques have
rarely been taken advantage of within
optimization with few exceptions, and
they are critical in acquiring domain
knowledge from a large amount of heterogeneous and noisy data. For optimization using meta-heuristics, such
knowledge is decisive in setting up a
flexible and compact problem representation, designing efficient search operators, constructing high-quality surrogates, and refining user preferences in
multi-objective optimization.
Second, how can we formulate the
optimization problem so that new
techniques developed in big data
research can be more efficiently leveraged? Traditional formulation of optimization problems consists of defining
objective functions, decision variables

Table of Contents for the Digital Edition of Computational Intelligence - November 2014