IEEE Computational Intelligence Magazine - August 2020 - 69

The results indicate that SSA can maintain a positive
strength correlation with the strength index, which enabled
professional players to choose an appropriate opponent
and provided game balancing, subsequently improving the
entertainment value and creating more diverse game play.
who wishes to do so on the two websites. On these platforms, there are
options to indicate openly to the community whether an account is AI or
human. To minimize irregular play
from human opponents, we did not
disclose this information. No surveys
to human players were conducted.
Since we do not have a set of reference Go ratings for the online human
opponents, we use the dan and kyu
ranks on Tygem and FoxWQ as the

indicator for actual program strength.
The online ranking mechanism (on
these platforms) is described as follows.
Each new account is given an initial
rank; subsequently her rank is adjusted
using the most recent n games. For
instance, a common setting would be to
promote the player's rank by one if she
has 12 wins in the most recent 20 games
against players with a similar rank, or
demote one rank if there were 12 losses.
Once a promotion/demotion occurs,

TABLE VII Normalized contingency table (column).
PERCEIVED STRENGTH

ENTERTAINMENT

1.00

0.03

0.13

0.24

0.14

0.04

0.02

0.13

0.12

0.29

0.08

0.05

0.07

0.08

0.13

0.14

0.08

0.14

0.09

0.08

0.06

0.29

0.17

0.27

0.24

0.06

0.25

0.17

0.15

0.12

0.06

0.12

0.14

0.25

0.19

0.24

0.31

0.13

0.06

0.03

0.05

0.14

0.19

0.06

0.13

0.11

0.15

0.18

0.19

0.35

7
6
5
Rank (Dan)

asked participants about the perceived
strength of the opponent, entertainment
level, and any other general comments.
We first annotate general comments
before analyzing numerical survey
results. In regards to game balancing and
perceived strength:
❏ P4: White (CGI)'s moves at 48 and
50 are interesting, from 58 to 66,
white played wonderfully, winning
the game at 94.
❏ P6: This version does not seem to be
too strong (z = 1.1). This one is
stronger (z = 1.4).
❏ P8: It feels like this game (z = 2.5)
has a sizeable gap in strength (stronger) compared to the previous game
(z = 2).
❏ P9: The moves at 20, 24 (CGI) are
plays that have not been seen in any
human game records.
In terms of quantifiable survey
results, we construct a normalized contingency table, as shown in Table VI.
The wording on the survey instructs
the player to give a perceived strength
rating from 1 to 9 (weakest to strongest), relative to her own playing
strength. First, we can see that in terms
of entertainment value, professional
players tend to enjoy opponents that are
slightly stronger than themselves (for
entertainment scores of 7 to 9). For
opponents that are much weaker than
the players, the entertainment value is
rated worst with no exceptions. In
terms of strength, it is interesting to
observe that for the strongest relative
opponents (perceived strength of 9),
there are two extremes. Some find
highly challenging opponents to be
very entertaining, while others do not
enjoy playing against opponents that are
far stronger than themselves.
Next, we collect data for amateur
players by collaborating with the two
largest online Go platforms (in terms
of players): Tygem [28] and Fox Weiqi
[29] (abbr. FoxWQ for the remainder
of this paper). Both websites provide
AI developers with toolkits for interfacing upon request. A total of 14 differently strength-adjusted programs
(7 each for dan and kyu ranks) were
available to play against, for any player

4
3
2

Tygem
FoxWQ

1
0
-1.0

-0.5

0.0

0.5
(z )

1.0

1.5

2.0

FIGURE 8 Dan rank strength correlation to z for the online Go platforms Tygem and FoxWQ.
0 dan corresponds to 1 kyu.

AUGUST 2020 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - August 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2020