IEEE Computational Intelligence Magazine - August 2020 - 66

IV. Dynamic Strength
Adjustment (DSA)

As stated in the previous sections, this
paper presents a flexible strength adjustment method simply by altering the
value z with an appropriate R th, say 0.1.
Moreover, the strength ratings are
approximately linear with respect to z in
the interval [- 2, 2]. This allows us to fit
the program's strength to its opponents'
dynamically, provided the opponents'
strengths are within Elo rating differences
of [- 983, -153] (compared to the original ELF), corresponding to the range
of z in [- 2, 2] (shown in Figure 2,
R th = 0.1). This section introduces two
types of dynamic strength adjustment,
inter-game and intra-game strength
adjustment. For the former, strengths are
adjusted between multiple games based
on previous game results, while for the
latter strengths are adjusted within each
game. We present two methods of
dynamic strength adjustment (DSA) to
showcase how we can predict opponent
strengths and adjust accordingly with rel-

ative ease. There are many ways to design
DSA mechanisms; the presented methods
are by no means a comprehensive review
of all available methods.
A. Inter-game Strength Adjustment

Inter-game strength adjustment is relatively easy. Namely, the strength index z
of a game is adjusted based on the previous game results and the index remains
unchanged within the game.
In this section, a simple adjustment
method is presented and demonstrated
to predict the opponent's strength. The
prediction can then be used to set z
accordingly. The strength index z is
decreased for every win and increased
for every loss, both by a small amount
Tz. The initial value of z is set to 0. The
value Tz is initialized to Tz init and
decreased by a discount factor r for each
game, capped by a lower bound Tz low.
In our experiments for the method,
Tz init is 0.375, approximately equivalent
to 100 in Elo rating based on the linear
regression in Figure 1, then decreased by

z=2

Strength Index (z )

2
1

z=1

z=0

−1

z = −1

−2

z = −2

40
60
80
Number of Games

100

FIGURE 5 Strength index estimation for inter-game SA.

TABLE III Win rate (WR) and average dynamic strength index (Avg. z) against
different opponents using inter-game SA.

OPPONENT

Z=2

Z=1

Z=0

Z = −1

Z = −2

W/O DSA WR

5.6%
(±2.9%)

8.8%
(±3.6%)

50.0%

78.4%
(±5.2%)

87.6%
(±4.2%)

INTER-GAME WR

43.4%
(±4.4%)

46.6%
(±4.5%)

52.0%
(±4.5%)

50.0%
(±4.5%)

54.8%
(±4.5%)

AVG.Z

1.93

0.88

-0.04

−1.06

−1.73

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2020

a factor of r = 0.95 for each game, with
Tz low = 0.03, approximately equivalent
to 8 in Elo rating. In the experiment,
100 games are played against each of the
five opponents whose strength indices
are z = 2, 1, 0, -1, - 2 for a total of
500 games. The experiment is repeated
five times and the following experimental results are based on the average values
of the five times.
In Figure 5, each of the five lines
indicates the predicted z for each opponent. The result shows that our method
can approximately predict opponents'
strengths and clearly distinguish five
opponents after 10 games. Table III also
shows that the averaged win rate for
each opponent is within 43% + 54%
and the averaged predicted z is very
close to the opponent's.
B. Intra-game Strength Adjustment

Intra-game strength adjustment is relatively challenging, given that the algorithm only has one game to predict the
opponent's approximate level of play.
Players often play inconsistently, mixing
objectively good and bad moves within
the same game. On the one hand, adjusting by large amounts leads to high variance of program strength. On the other
hand, if strengths are adjusted by a small
amount, the effects may not be sufficiently obvious.
Our method is as follows. In principle, we still attempt to maintain all
moves so that the overall win rate is
around 50%. For each move, we first
estimate the current win rate W, by
using the MCTS win rate of the move
with the most simulation counts. The
index z is decreased when W 2 50%
and increased when W 1 50%, both by
a value of Tz. The program chooses
moves based on the softmax policy proposed earlier.
For stability, Tz is set to be relatively
small when W is within a range (50% - e,
50% + e) where e is a user-defined value,
say 10%. Namely,
Tz =
TZ ,

TZ #

if |W - 50%|$ e
|W - 50%|
e

, if |W - 50%|1 e.
(18)

IEEE Computational Intelligence Magazine - August 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2020