IEEE Computational Intelligence Magazine - August 2020 - 65

g
rel

g

D

0.1
Enorm
(z )

where R (i j ) = N (i j ) /N (1j ). Moreover, let
the relative rating be normalized to be
independent of the value H as follows.
E

-32

−200

-64

−400

-96

−600

-128

−800

-160
-174

−1,000
−1,088

g

E ( z ) = E ( z ) - E (3 )
= c H m/ / r (i j )(z)( ln R ( j ) ), (12)
mg j i

g
norm

0

0

g
rel

(z) = E (z) /H
= c 1 m/ / r (i j )(z)( ln R ( j ) )
mg j i
= E g ;/ r (i j )(z)( ln R (i j ) )E .

−4

−3

−2

(13)

g
D 0.1
E norm
(z) = E g + D0.1[E norm
(z)]

= E g + D0.1;E g ;/ r (i j )(z)( ln R (i j ) )EE
= E g + D0.1;/
i

i

(z)( ln R (i j ) )E .

( j)
ri

(14)

Figure 4 depicts the solid curve of
0.1
E Dnorm
^z h calculated from the set D 0.1
according to Equation (14). The left
0.1
y-axis indicates the value of E Dnorm
^zh .
The curve resembles a logistic function.
Now, let E D0.1^zh denote the expected
rating, and H D0.1 be the value H, under
the set of games D 0.1. Thus, we have
0.1
E D0.1(z) = E D0.1(3) + H D0.1 E Dnorm
(z). (15)

Then, we can derive that
0.1
E D0.1(- 3) = E D0.1(3) + H D0.1E Dnorm
(- 3),
(16)
and

3

4

Empirical
Logistic Regression

i

For stochastic analysis, we extend by
collecting some sets of games, each of
which is collected from the games under
a designated threshold ratio in the above
empirical experiments. We exclude extreme cases to minimize the effect of
noise for our analysis. For example,
t h e cases of z = 3 and z = - 3 are
not included.
For simplicity of analysis, let us illustrate the case for D 0.1 , denoting the
set of games with threshold ratio 0.1,
which contains about 2000 games. The
expected relative rating under the set
D 0.1 is

−1 0
1
2
Strength Index (z )

Elo Rating (Relative to ELF)

In the above formula, the first item
is fixed, and therefore can be omitted
when calculating ratings relative to the
one with z = 3, similarly, as follows.

D

0.1
Enorm
(z )

0. 1
FIGURE 4 The curve of E Dnorm
(z) and the empirical data.

H D0.1 =

E D0.1(- 3) - E D0.1(3)
.
0.1
E Dnorm
(- 3)

(17)

S i n c e t h e v a l u e s E D0.1(3) a n d
E (- 3) are supposed to approximate
the strength in the empirical experiments,
they can be replaced by the empirical
strengths at z = 3 and z = - 3, whose
relative Elo ratings are 0 and −1088 as
shown in Table I. Thus, the value H D0.1 is
derived to be 6.243 according to the
above formula. The right y axis in Figure 4 follows the y axis in Figure 1. The
regression error to the empirical strengths
for z between −2 and 2 is about
40.45 Elo, and the regression error to a
logistic regression curve is about 10.51
Elo ( b 0 = - 0.05, b 1 = 1.17). These low
errors justify the hypothesis.
In our experiments, we also derived
the value H for other sets of games, as
shown in Table II. From the table,
H D0.25 is almost the same as H D0.1, while
H D0.5, H D0.05 and H D0.02 are lower. For
H D0.05 and H D0.02, our conjecture is that
the noise incurred from having a low
threshold ratio is high as the following
illustration. In the case of R th = 0.02,
since the average number of simulation
counts for the best move is about 259.4
D 0.1

TABLE II The conjectured strength
indices estimated in different data sets.
DATA
SET D0.02
H

D0.05

D0.1

D0.25

D0.5

4.385 5.369 6.243 6.244 3.292

with the one second time limit, it is
highly likely to include the moves with
very low simulation counts (the threshold is about 259.4 # 0.02 , 5.2). Since
many of these simulations may be generated simply because of the exploration
bias, these simulations may introduce
noise and therefore affect the verification of our hypothesis.
As for H D0.5, we observe that the
average number of candidates is 1.4
from Figure 3. Since the number is relatively low in many cases, the policy
chooses only from a single candidate
move. Therefore, the distribution is
insufficient to justify our hypothesis. As
an example, the most extreme case is
where the threshold ratio is 1, and only
the moves with the highest simulation
counts are chosen, as in the original
MCTS. The value of H in this case does
not affect the policy at all, since there is
only one choice.

AUGUST 2020 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

65
IEEE Computational Intelligence Magazine - August 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2020
