AlphaGo’s Mastery Continues

Wuzhen, China. The last two days have seen Google DeepMind’s AlphaGo Master continue its declaration of superiority by defeating Ke Jie (9 dan professional) for a second time and gaining victory in a hard-fought battle against a whole team of five Chinese professionals, all of them 9 dan and representing the very pinnacle of ability.

Can nobody defeat Dr Hassabis’ robot?

One feature of AlphaGo’s play stood out above all: it is weirdly comprehendable.

I have seen more than a few professional game records and, without commentary, the fighting at the very top level often comes across as a kind of black magic but AlphaGo’s moves seem natural and rhythmical and their meaning is often immediately apparent even when they’re placed on points that no human player would find.

The next most unbelievable observation is that the engine plays some of our human joseki.

Instead of inventing new ideas, AlphaGo is simply showing us how to better employ ideas that have studied for hundreds of years. This is amazing because it shows us that the research of us mere mortals has been astoundingly accurate. Our romantic heuristics with which we reason and argue are somehow correct – sub-optimal they might be but they are not entirely wrong.

How is it possible that such a simple game played with primitive material and a mere handful of rules can provide a space in which such simple sequences can have so many layers of meaning and possibility?

I wonder what the future holds for AlphaGo and the game of Go.

We have seen AlphaGo discredit a lot of conventional wisdom that is taught like a mantra from the early kyu ranks, in the schools of the Insei and in professional study sesssions alike. To many, this opening of the mind has been a liberating experience comparable to the revolution in opening-theory pioneered before World-War II by Go Seigen and Kitani Minoru: the Shinfuseki.

“You have sente; you can play anywhere you like!”

I would like to ask AlphaGo many questions. I lost a game, once, to a rival who played what is known as the Great Wall opening – a crazy thing with a wavy line of stones down the middle of the board. I scoffed; I played high in all four corners and ended up a few points short of his total. What would AlphaGo think of such unconventional opening positions?

How strong would AlphaGo proove to be if you mandated that it starts at tengen – the point at the very centre of the go board?

Would AlphaGo find Honinbo Shusaku’s “ear-reddening” move?

How would AlphaGo handle a thousand-year ko?

I think that the game of Go, played between humans, will survive this onslaught by artificial intelligences simply because human play is sub-optimal. AlphaGo has shone a light on our imperfection and, honestly, it has been a painful experience but precisely this imperfection will be what saves the game.

The fact that a human opponent will never play perfectly, however strong they may be, allows opportunities for creativity. The fact that artistry governs a human’s opening and early game more than hard numbers or utility-scores allows a chance for exploration, experimentation and poetry.

If there is one thing to fear it is a future in which human professionals rely on bots too heavily, learning to mimick them and refusing to consider sequences that have not been ‘legitimised’ by some algorithm’s play. This is not to be feared because it will change the nature of competition. It will not lower the level of play between top players. It will merely be boring and bland.

Tomorrow, Ke Jie is to play a final match against the machine and he has politely requested to take the white stones – a request that DeepMind graciously granted. (Lee Sedol did the same in 2016.)

I have no doubt that tomorrow’s game will be spectacular but, despite the fact that both Ke Jie and AlphaGo appear to believe that playing white is advantageous, I think the best our human champion can hope for is a spirited performance and an honest loss in good style.


AlphaGo vs. Ke Jie: Game One

Yesterday, on the first day of the Future of Go Summit in Wuzhen, China, Google DeepMind’s AlphaGo (9 dan professional) neatly defeated Ke Jie (9 dan professional), the nineteen-year-old ‘final hope’ for human players of the ancient game. I watched the live-stream and saw the thing unfold.

I have been playing Go for many years and I am currently ranked in the middle of the “single-digit kyu” section of the amateur rank table in both Europe and South Africa. I watched, mused and wrote while last year’s Google DeepMind Challenge Match saw the upstart’s robot beat Lee Sedol (9 dan professional) – the Roger Federer of Go. In March 2016, this was a huge surprise to me as a computer scientist and Go player: it was too soon. It was a decade too soon!

I watched as Deep Zen – another of these new-generation of deep-learning-powered hybrids – failed to repeat DeepMind’s achievement when matched against Cho Chikun (9 dan professional) – he of the crazy hair, the tea drinker, almost definitely the most well-known Go player alive.

I read the stories and perused the records from the sixty games played online by the mysterious character who identified as Magist or Master and was later revealed to be none other than a new revision of DeepMind’s AlphaGo. The engine won fifty-nine of those games – all sixty if you grant her a victory for one game that ended prematurely due to technical troubles on the human-player’s end – but, by the end of the year 2016, this outcome was a foregone conclusion in my mind.

From all the evidence, it appears that AlphaGo has walked the path from first victory by an algorithm against human champion to last victory of a human champion against an algorithm in about one year. In the Chess world, this took a decade or more.

I might be wrong. Tomorrow, Ke Jie might strike back – I hope he does. It seems unlikely.

Ke Jie (9 dan professional) and AlphaGo’s handler – I think thought he was Fán Huī (2 dan professional) but he was actually Aja Huang (5 dan amateur) – took their places at the game-board and, with a few official statements by the coordinators and the formality of nigiri, the colours where assigned and the game began. Ke Jie placed the first, black stone at the three-four point: komoku.

The first surprise of the game was Ke Jie’s second move: the three-three point in the corner opposite to his first. The second was white’s creation of a two-space corner-enclosure with both stones high, on the fourth-line. Ke Jie invaded a white corner insanely early and white played a cut that seemed, at first, to be a little senseless. The game was certainly not boring in any way, although none of these anti-traditional plays appeared crazy to the eyes of the audience: amateur and professional players, alike, who had had their eyes and minds opened to such new possibilities by previous AlphaGo games.

The fashion in which AlphaGo calmly but inexorably drew ahead was reminiscent of last year’s event. There were no major upsets. Trades and exchanges of territory happened without drama but they are to be expected at such a high level. By the end-game, white’s advantage was so marked that she could afford a few inefficiencies for the sake of certainty and safety.

The result was another victory for the software: white won by half a point.

Numerically, half a point is the smallest margin for victory that the rules of the game permit and such an outcome is only possible for reasons of tradition and aesthetics: a draw would be considered undesirable and so white is granted the half-point as a tie-breaker. Despite all that, this was a very comfortable victory for the bot.

You could see evidence of the bot’s confidence of victory in the moves that were played during the latter stages of the game as well as the moves that were not. Ke Jie was the one who seemed desperate to claim super-optimal points while AlphaGo seemed content to capture stones that might present a later risk and to sacrifice a point here or there to ensure safety over all.

After the game, during the atrociously translated press conference, Ke Jie praised AlphaGo as a deity, stated his will to learn from the bot and, somewhat contradictorily, mentioned that this event will be the last time he plays against an artificial intelligence.

Ke vowed never again to subject himself to the ‘horrible experience’,” wrote The Guardian.

The Guardian and many other western news-sources painted Ke Jie as a sore loser, siezing his words and quoting them next to an unfortunate photograph, both out of context. They called him glum and, if one did not know better, one might simply swallow such a tale.

Go is a intense, immersive and deeply emotional game even at my middling amateur level. I can only believe that it is more so at the top. Horror and displeasure are emotions. Humans feel them. I am sure that all professionals have had moments of horror in many games – perhaps in all their games. Not for a moment will I accept that Ke Jie was complaining about his plight.

I think he was actually talking in general about the experience of playing against a machine – a stance that is further supported by the fact that he went on to say that he found games against other human beings to be more rewarding and more enjoyable.

I agree with Ke Jie on this count. Playing against another human player adds emotional, psychological and social dimensions that are simply absent from machines and are even somewhat lacking when playing across the Internet. Between two humans, Go is a sharing of mental creativity and an intimate communication that has nothing to do with words or language.

It is also less of a blow to lose to another living person than to a machine or a nameless avatar. Be they your sworn enemy, life-long rival or simply a personality who grates on your nerves, there is still some camaraderie, good-will and friendship.

I will watch tomorrow’s game, eagerly, and I am seriously intrigued by the pair-go and team matches that will follow on Friday and Saturday but, for me, the novelty of this style of contest is starting to wear thin.

I feel there is no artistry in the Go that is played by the algorithm. The beauty and poetry is missing. That it approaches optimality is not in question but optimality is somewhat boring.

When a master studies the record of a game he can tell at which point greed overtook the pupil, when he became tired, when he fell into stupidity, and when the maid came in with the tea.” – anonymous.

Robots neither drink tea nor require rest. They feel no emotion and no such impulse from their opponent will flavour their actions.

AlphaGo vs. Lee Se-dol: Game Five

Lee Se-dol resigned before the last handful of “yose” or end-game moves had been played in this morning’s game, bringing the Google DeepMind challenge match between AlphaGo, their Go-playing programme, and the 9 dan professional human player to a close. The artificial intelligence won the match by defeating Lee Se-dol in four of five games.

Lee Se-dol took the black stones and began with a solid, dignified opening that emphasised territory. He resisted white’s attempts to disrupt his plan and denied white the time to reinforce the boundaries of the areas of influence that white acquired in exchange. When a large group of white stones was cut off by a sequence that involved the infamous “stone tower” shape, it seemed like black was in the lead – he had gained thickness and resolved several weaknesses in exchange for a tiny number of points.

Slowly but surely, with calm and sensible late middle-game and end-game fighting, AlphaGo turned the game around once more and by the time the temperature of the end-game moves had fallen to nothing more than one point a move, it was apparent that black would be behind.

Lee Se-dol’s resignation was clear proof of his finely honed ability to count the score in his head because his deficit was barely measurable.

During the pre-match discussion, Google DeepMind talked about the fourth game of the match, the game that AlphaGo lost. They explained that move 78, the tesuji or skilful play that precipitated the demise of AlphaGo in that game, had “surprised” the engine and forced it to build a new plan for a branch in the game tree that it either hadn’t seen or had only explored perfunctorily. A human player in a similar situation would take a lot of time to re-evaluate and recalculate but AlphaGo’s time-control strategy is apparently very simple and primitive and it neglected to invest much of the abundance of time available on its clock.

Today, the post-match press conference was handled ineptly, the English translation coming and going and generally chaotic. A few interesting points could be gleaned from the chaos, including the fact that an ethics committee was set up inside DeepMind as a condition of their sale to Google.

Lee Se-dol also took ownership of his performance once again, saying that he believes that humans can do more against A.I. Go engines and admitting regret that he was unable to show us how. He reminded the audience of the importance of human creativity, said he began to question some classical beliefs about the game “a little bit”, and indicated that he had more studying to do.

AlphaGo’s skill with the stones is not superior to that of top human professionals, according to Lee Se-dol, its advantages are concentration and the psychological facets of the game.

During the ceremonies that followed the game, AlphaGo was awarded an honorary rank of 9 dan by the Korean Baduk Association.

This match is over but its culmination should be considered a beginning rather than an ending. Google DeepMind have executed a historic début, winning five-nil against Fan Hui (2 dan professional) towards the end of 2015 and four-one against Lee Se-dol, but AMD and Facebook and other contenders are yet on their way to the party. AlphaGo developed its strength through self-play – what will happen when these learning algorithms start to train against each other, like human insei, and new blood-lines are introduced to this gene pool?

In the coming months, we will learn more details about what happened in these five games. We will learn what Google DeepMind plan to do with their creation and how it will impact the worlds of Go, artificial intelligence and machine learning. We will see how Lee Se-dol moves forward in his own, human career. We live in exciting times.

AlphaGo vs. Lee Se-dol: Game Four

Lee Se-dol obliterated Google DeepMind’s AlphaGo with an inspired ‘wedge’ in this morning’s game, the fourth game of the challenge match between the 9 dan professional human player and the upstart A.I.

2016-03-13 __go4go_20160313_Google-AlphaGo_Lee-Sedol0000
The moves leading up to Lee Se-dol’s incredible move: white 78

Absolutely everyone showered praise on Lee Se-dol for his fantastic play, among them, Gu Li, a 9 dan professional player from China who is considered to be one of the strongest professional players on the planet and particularly relevant because of the international rivalry that exists between him and our champion facing DeepMind’s engine, today.

After the game, Lee Se-dol was finally ready to tell us where the weaknesses lie in this version 18 of the distributed AlphaGo programme. He said that the A.I. struggled, holding the black stones, and said that “surprises” like his skilful wedge in the centre forced “bugs” to show in the bot’s play.

He even put his money where his mouth is by explicitly requesting to play as black in Tuesday’s match. Presumably, now that he has identified a vulnerability, he intends to show the world that he can defeat AlphaGo even when it is playing its preferred role in the game.

During the press conference that followed the game, one reporter raised concerns that AlphaGo’s database of professional game records equipped it with extensive knowledge of Lee Se-dol but, to his disadvantage, the man knew almost nothing about the machine. He dubbed this imbalance “information asymmetry” and, in response to his question, Demis Hassabis, one of the founders of Google DeepMind, made some intriguing statements. Firstly, Mr Hassabis stated that AlphaGo’s training database did not contain any game records from professional games played by Lee Se-dol. He said that its database was populated with amateur dan-level games only and, from there, AlphaGo had trained by playing against itself. He pointed out that even a thousand records of real-world games would be insignificant amongst the millions of records created by self-play. What he says makes sense but I do wonder why they didn’t prime the system with professional game records in addition to amateur ones — it seems like an easy thing to do.

In the first three games of the challenge match, AlphaGo exhibited a high level of skill and mostly played moves that made sense, even to us humans. It earned the respect of its opponent and the Go community in general. Lee Se-dol said it was not unreasonable; I was among those who praised it for its human-like play. When it was ahead, it played moves that could be considered sub-optimal but even those were not absurd. Today, things were very different.

Even after Lee’s wedge, all was not lost, but AlphaGo sealed its fate by playing appallingly once behind on the board — many of its moves were simply ludicrous — and spurned its hard-won good-will by charging blindly onwards when its situation had become hopeless and the only responsible act was prompt and polite resignation. Both of these behaviours are familiar to anyone who experienced the laughable death-throes of the Monte-Carlo Tree-Search Go engines and, unfortunately, today’s performance presented an ugly glimpse of AlphaGo’s pedigree.

Yesterday, we saw proof that AlphaGo has conquered the mystical art of Ko and that assures me that, one day soon, these bots will also be able to fight to the fore after falling behind and learn to resign with dignity but, for now, DeepMind’s ‘prototype’ still needs work.

AlphaGo vs. Lee Se-dol: Game Three

This morning, Lee Se-dol lost a third and deciding game in the challenge match against Google DeepMind’s AlphaGo. The fourth and fifth games will be played but the prize goes to the programme and the prize money will be paid to various benevolent causes.

The game started with a high ‘Chinese’ fuseki which was soon broken up by a vital battle that affected the whole board, a product of Lee Se-dol’s aggressive style and AlphaGo’s refusal to capitulate. Black fought for his heavy and inefficient stones but white, played by the bot, acquired a huge territory that black simply couldn’t match. In desperation, black invaded and a scuffle lead to a complicated situation with multiple Ko fights. Black resigned when it became clear that his stones were lost and, elsewhere on the board, the cost of the scrap became too great to bear.

One incontrovertible fact was proven during the match: AlphaGo does not struggle with Ko fights in the slightest. AlphaGo showed that it can handle extremely tricky Ko situations that play havoc with the game-tree and would have completely flummoxed its ancestors, the MCTS engines of yester-year.

Lee Se-dol admitted that AlphaGo defeated him, taking ownership of his defeat and lamenting that he did not show us a better game, today. He said that today’s defeat was his defeat, not a defeat of human beings.

Michael Redmond, the 9 dan professional westerner who provided the official English commentary during the match, brazenly declared that AlphaGo “beat Lee Se-dol at his own game!” In front of a veritable army of reporters, in the ballroom of a hotel in down-town Seoul, he went on to suggest that AlphaGo would herald a “third revolution” in Go opening theory, mentioning AlphaGo in the same sentence as Honinbo Dosaku and Go Seigen, two legendary human players who sparked extensive, novel innovation in the fuseki in the past.

When a member of the audience questioned Redmond’s prophecy, Lee Se-dol answered that he thought AlphaGo is not at the level of the so called ‘Divine Gods’ and described the bot’s play as ‘different’ and ‘superior at times.’

Meanwhile, in the Go community, speculators questioned whether Lee Se-dol (9 dan professional) was, in fact, the best player to champion humanity in this duel against the machine. Some pointed to his aggressive and frequently precarious fighting style and hypothesised that AlphaGo’s cold and heartless play was ideally suited to winning in such situations – situations in which mere mortal, human professionals might become flustered or overwhelmed by the complexity. Some suggested that calmer, more placid players might fare better against the A.I.

Lee Se-dol reassured the audience at the post-match conference that AlphaGo is not yet perfect. I believe him. If he does not defeat AlphaGo, another will, but I do hope that he shows us the way over the course of the next two games.

AlphaGo vs. Lee Se-dol: Game Two

Lee Se-dol (9 dan pro.) conceded another match to AlphaGo, today, in Seoul. Google DeepMind’s lead now stands at two games to none and a straight-sets victory over the human player seems entirely possible. We will have to wait until Saturday to find out.

During the press conference that followed the game, Lee Se-dol admitted that he felt that black, played by AlphaGo, never fell behind or gave him an easy chance at any point during the match and that the programme made moves that he considered entirely reasonable. Reading between the lines, I suspect he even felt some sort of respect for the thing! He attributed his loss to an inability to find the bot’s weaknesses.

Google DeepMind claimed that the A.I. itself was confident of victory for most of the game, particularly during the opening and end-game phases, but that the team behind the scenes were nervous because the opinions of other strong human professionals were widely varied – some predicting that white, Lee Se-dol, would ultimately come out in front.

The development team also claimed that their only role, during the game, was ensuring that the programme had access to the computational resources and infrastructure that it required.

In the coming months, I would dearly like to know more about the level of human intervention that took place behind the scenes, the computational resources that powered the engine, and the learning and improvement that took place between the games of the match.

Personally, I will be cheering for the man who is opposing the machine, on Saturday. AlphaGo has already acquitted itself skilfully and Google DeepMind have already made a smashing début; a close match will be far more satisfying than a clean sweep by the engine.

While we wait to see what transpires in game three, it is time to start considering what AlphaGo’s victories mean for the future of the game of Go itself, the future of professional, competitive play and the future direction of research in the field of Computer Go. Perhaps we should all start reading and remembering the ripples that IBM’s DeepBlue sent through the world of Chess.

AlphaGo vs. Lee Se-dol

In the wee hours of this morning, I watched Google DeepMind’s AlphaGo programme defeat Lee Se-Dol (9 dan professional) in the first game of a five game match that will continue for the rest of this week. They played on a full-sized (19 by 19) board, AlphaGo played white and Lee Se-Dol paid komi. The field was entirely level, the game entirely fair. This simply wasn’t supposed to happen during my lifetime!

When I first encountered the game in 2007, my friend and fellow programmer promoted it as a game that bots simply couldn’t play. At that time, GNU Go was available but barely stronger than an improving double-digit kyu journeyman like myself – I soon learned to defeat it. A few bots exploited Monte Carlo Tree Search (MCTS) and they could achieve low single-digit kyu strength on a regular desktop computer or amateur dan strength on exceptional hardware but that was about all.

Everyone knew that MCTS was the way forward, that heuristics used in the random Monte Carlo simulations would improve with time, and that the inexorable increase in exploitable computational power would permit an ever-increasing number of simulations to be run. Everyone knew that even the professionals would fall, one day. That day was supposed to be decades away and the DeepMind team proved everyone wrong.

This morning, the outcome of the game was far from the only impressive feat that DeepMind’s programme demonstrated: the human-like nature of its moves and the mature management of the clock were a wonder to behold, as was the calm and steadfast way in which it handled Lee Se-dol’s cunning but unorthodox opening that seemed like an attempt to deviate from more common and well-known openings that would exist in databases and opening books that the artificial intelligence was surely using.

I was strangely sad when I saw Lee throw in a prisoner to indicate his resignation, this morning. In that instant, something ineffable changed in a way that can never be reversed. I will continue to watch in fascination as this match unfolds but, even if Lee wins the four remaining games, history was made this morning and I am very happy to have watched it as it happened.