Computational Intelligence - February 2017 - 45

GA Is Applied in
Niches Defined
by [M ]

Subsumption

GA
Explore

Environment

Input
Feature
Vector

Executed
Action

Random
[P ]
Current
Classifier
Population

Form Matchset

[M ]

Select Action

Set of Classifiers
Matching Current
Input

Covering

Exploit

Covering Operator
Creates New Matching
Rules When [M ] Is Empty
Form Actionset

[A ]

Reward
Signal

Prediction
Array

Set of Classifiers
Advocating
Selected Action
Parameter
Update

[A ]-1

Last
Actionset

XCS Operation
Figure 2 Working of XCS in reinforcement learning tasks. The gray-shaded boxes represent different classifier sets used in the system and the
blue-shaded ellipsoids represent different processes that take place in the system.

alternate between a random action
selection, encouraging exploration, and
action selection based on the prediction
strength of the matching classifiers, to
exploit current knowledge. Finally, the
fitness of all rules in the [M] is updated
based on the reward (e.g., 1000 for a
win, -1000 for losing a game, and 0 otherwise) received from the environment
for the selected action. A GA cycle is
applied on the classifier population in
either [M] or [A] based on the average
experience of classifiers in either set.
In Pittsburgh-style LCS, a complete
rule set constitutes a classifier. Hence, the
system works on a population of variable-length rule sets using a generational
GA, in which each rule set is considered
a potential solution to the problem. The
rule evaluation is performed in a batchmode over the entire data set (or a sufficiently large number of examples), and
the genetic operators are applied at the
rule set level at the completion of a rule
evaluation cycle. In other words, the GA

in these systems works more traditionally as an optimization heuristic to
evolve the best solution in the form of a
complete rule set. Since in this set-up
the solutions require longer evaluation
trials to measure their quality, these systems are more suitable for offline tasks
and often are applied to concept learning problems using a supervised
approach. Incremental variants of these
systems which are potentially better candidates for use in computer games, have
appeared recently [46]. However, this
area is largely unexplored.

the relationship between game theory
and computer games [47], [48] have
been acknowledged in the literature.
Moreover, as the use of computer games
expands beyond recreation (e.g., in education [49] and management [2], [3] sectors), game-theoretic models could play
an even greater role in game design. For
these reasons, we included LCS-based
game-theoretic studies in this review.
Overall, this review has been organized
into four types or categories of games,
which are discussed below.
A. Video Games

IV. Types of Games

Broadly, the LCS approaches included in
this survey can be categorized as those
related strictly to computer games and
to game theory. Computer games, such
as video and board games, are modeled
and played in-silico. Game-theoretic
studies are generally not categorized as
computer games because of their mostly
non-recreational applications. However,

A variety of video game genres exist
[50]. This section provides an overview
of the famous genres, with a focus on
the types relevant to the LCS work in
games. Among these, action games are
one of the earliest and well-known
types of video games, with a human
player generally controlling and maneuvering a character in a given environment. Some of the famous action game

FEbruary 2017 | IEEE ComputatIonal IntEllIgEnCE magazInE

Table of Contents for the Digital Edition of Computational Intelligence - February 2017