IEEE Computational Intelligence Magazine - August 2019 - 13

C. Supervised Strategy Selection Learning

3) Results
The resulting policy network has an accuracy for predicting the
correct puppet move of 73%, and a 95% accuracy for predicting
any of the top 2 moves. Table III shows experimental results

100
90
80
70
60

00
4,

60
3,

80
2,

40
2,

00
2,

Neural Network
Lanchester
Optimized Eval.
Simple Eval.

3) Results
The evaluation network reached 95% accuracy in predicting
the winner of game states from the test samples. Figure 3
shows the accuracy of different evaluation functions as game
time progresses. Table II shows the performance of the Puppet
Search adversarial search algorithm when using the Lanchester
evaluation function and the neural network. The performance
of the network is significantly better (p-value = 0.0011) than
Lanchester's, even though it is three orders of magnitude slower. Evaluating a game state using Lanchester takes an average of
2.7μs, while the evaluation network uses 2,574μs. LightRush
and HeavyRush are used as baseline scripts and their descriptions can be found in [28].

2) Training
Training is analogous to the previous network, but labels are
now a choice of strategy rather than win/loss. To generate the
labels we executed a 10 second Puppet Search on the same
positions we used to train the evaluation network.

Accuracy

risk not having enough data for your network to generalize properly.
We used the set of bots shipped with the μRTS system
and 5 different maps, each with 12 different starting positions.
Ties were discarded, and the remaining games were split into
2,190 training games, and 262 test games. These test games
were used to validate the network's ability to generalize to
previously unseen data. Twelve game states were randomly
sampled from each game, for a total of 26,280 training samples and 3,144 test samples. Several evaluation functions were
trained, for comparison, on the same dataset: the evaluation
network, a Lanchester attrition law based function [18], a simple linear evaluation with hard-coded weights that comes
with μRTS, and a version of the simple evaluation with
weights optimized using logistic regression.

Game Frame
FIGURE 3 Evaluation accuracy of different value functions in μRTS.
The accuracy of predicting the game winner is plotted against game
time. Results are aggregated in 200 simulation frame buckets.
Shaded areas represent one standard error.

TABLE II Evaluation network versus Lanchester: round-robin

tournament using 60 different starting positions per matchMost adversarial AI systems in commercial video games are
up and 100 ms of computation time. Values indicate win
scripted-or hand-authored-by designers. Finite-state
percentage of row vs column agent.
machines, behavior trees and decision trees are the norm. In
PS
PS
LIGHT
HEAVY
some game genres, such as first-person shooter games, these
CNN
LANC.
RUSH
RUSH
AVG.
scripts control a single unit, while in others, like RTS games,
PS CNN
-
59.2
89.2
72.5
73.6
they control all player's units. In both cases, several scripts can be
PS LANC.
40.8
-
64.2
67.5
57.5
used to implement different strategies, and realism and difficulty
LIGHTRUSH
10.8
35.8
-
71.7
39.4
can be raised by smartly choosing when to switch between
them [8]. Based on this work, here we will train a neural netHEAVYRUSH
27.5
32.5
28.3
-
29.4
work to recognize which script,
among four basic ones available in
μRTS, is the best choice for a given
TABLE III Tournament results of the policy network versus Puppet Search and fixed
game state.

1) Architecture Changes
Our evaluation network was a classifier charged with choosing between
two categories: win or loss. The first
change we need in this context is to
have one output per possible scripted
strategy. The second change is to add
an extra input plane to indicate for
which player should the network
compute a strategy.

strategy scripts that produce certain unit types and send them to attack. Reported are
round-robin results using 60 different starting positions per match-up. Values indicate
win percentage of row vs column agent.
POLICY
NET.

LIGHT
RUSH

HEAVY
RUSH

RANGED
RUSH

WORKER
RUSH

AVG.

POLICY NET.

44.2

94.2

71.7

100

61.7

61.9

55.8

87.5

66.67

91.7

93.3

65.8

LIGHTRUSH

5.8

12.5

71.7

100

48.3

HEAVYRUSH

28.3

33.3

28.3

100

48.3

RANGEDRUSH

8.3

100

18.1

WORKERRUSH

38.3

6.7

7.5

AUGUST 2019 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - August 2019

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2019