Computational Intelligence - August 2016 - 70

value network, which was trained
through RL with 30 million self-play
game positions. The training process takes
about one week with 50 GPUs, for a
total training time of four weeks and a
day for all three networks.
The most time-consuming and most
difficult process to reproduce, however, is
not the training of these three networks,
but the generation of self-play game positions. For each of the 30 million self-play
positions, 100 playouts are performed; for
each playout, we assume on average 200
moves until game completion, so a total of
600 billion move data samples need to be
generated to train the value network. Let
us assume, for the sake of demonstration,
that a research team has access to four
GPUs. The training of the three networks
will take [(4 weeks # 7 days / week) +
1 day] # (50GPUs) / (4GPUs) = 362.5
days. Assuming a processing speed of 720
moves/s for a single GPU (with a batch
size of 16), an optimistic estimate for the
generation of self-play game samples is
600 # 10 9 moves / (4 # 720 moves /
second) , 208 million seconds, which
works out to 2411 days or about 80
months. In addition to the total time
required for generating and training the
networks, we must consider the fact that
parameters involved in the entire process
are rarely tuned to fit the requirements in
a single trial. This includes a wide variety
of settings such as the number of layers

and neurons in the neural network, the
features to use for the Go positions, the
collection of expert game records that are
used to train the initial SL policy network,
etc. This quick estimate of required
resources does not even take into account
the knowledge and experience that the
DeepMind team has acquired since its
inception. As a side note, the distributed
version of AlphaGo uses 280 GPUs [14].
III. Human Intelligence View

Demis Hassabis, CEO and Co-Founder
of Google DeepMind, described Go as
the "Mt. Everest" of AI [15] because Go is
a very complex board game that requires
intuitive, creative, and strategic thinking [16].
In the past decade, the techniques of
MCTS had revolutionized the field of
computer game-playing. The playing
strength of computer Go programs has
progressed to about a four-stone handicap
against top professional Go players in
2012-2015. More precisely, until AlphaGo's emergence in Oct. 2015, the world's
strongest program, Zen, was able to beat a
top professional Go player with a handicap of four stones, while losing when the
handicap was decreased to three stones.
Google DeepMind introduced a new
approach that combined MCTS with
deep learning in their program AlphaGo
[14], which subsequently broke this fourstone handicap barrier in the recent competition with Lee Sedol (9P) in Korea, in

Mar. 2016. AlphaGo's performance sent
shock waves through the community of
professional Go players and AI researchers.
Since then, people on the Internet and
media, particularly in Go-playing cultures
such as in Korea, -Taiwan, Japan, and
China, were buzzing with discussion on
related topics [17]-[19]. In this section, we
invited three high-level professional Go
players who have spent time helping the
development of computer Go programs,
including Coldmilk and MoGoTW, and
who have also been part of shaping Go
trends for many years, to comment on the
game results of the challenge match from
Mar. 9 to Mar. 15, 2016. Fig. 3 shows the
game record for match No. 5 and
-Comment 1 lists three professional Go
players' brief commentaries. For the other
four matches' commentaries and the full
commentary on match No. 5, readers can
refer to the online version of this article
[8]. Additionally, readers may find details
for the Go terminologies used in Comments 1 and 2 at Sensei's Library [20].
From the perspective of professional Go
player (Ping-Chiang Chou/6P), the
strengths and weaknesses of AlphaGo are
listed in -Comment 2.
IV. Discussions and Future Studies

It has been estimated that when human
players play against a new computer program for the first time, their strengths are
usually weakened by about 1-2 levels. This

Comment 1. Professional Go Players' Comments on Match No. 5
Comments by Ping-Chiang Chou (6P/Taiwan)
Theoretically, Black made a profit from the fight in the bottom
right corner. Hence, up to White 68, Black's situation was slightly
better from the view point of professional players. Chou concluded that perhaps Black 79 was decided under the premise that
Black already had the lead. If the match was against other professional players under the same situation, Black 79 would have
intuitively been played at L14. In reality, from White 80 to White
90 (jump), the situation had turned favorable for White.
Comments by Chun-Hsun Chou (9P/Taiwan)
White started to play an unusual move at move 18. White 20, 22,
and 24 are indicative of AlphaGo's unconventional playing style,
which sacrifices a few stones to gain a sente and obtain the
maximum profit. Black 79 and 81 were two slow moves intended

IEEE Computational intelligence magazine | AUGUst 2016

for stability. Unfortunately, they were shut in by White 80 and 82
so the situation favored White by a little. Although there were
variations in the following moves, White took hold of the situation with a weak superiority until the endgame. As a result, Black
ended up losing by about 2.5 points after komi 7.5.
Comments by Ming-Wan Wang (9P/Japan)
Shortly after the game began, White again made mistakes, allowing
Black to have the upper hand. This unexpected turn of events
caused Black (Lee Sedol) to change his playing style so that he may
conservatively aim to make life. After being laid siege by White,
Black realized he was behind, and started to make every effort to
regain his lead, whereupon the situation became quite chaotic. It
was unfortunate that Black did not play the strongest move at the
critical moment, which ended the game with a small loss for Black.

Table of Contents for the Digital Edition of Computational Intelligence - August 2016