A Good Chess Cheater Might Never Be Caught

Ever since he beat the greatest chess player who ever lived, Hans Niemann has been called a cheat. The 19-year-old’s surprising victory over Magnus Carlsen in St. Louis on September 4 led to accusations that he’d been taking cues from a chess-playing AI, or chess “engine.” Niemann later admitted to having done just that on two occasions—both times when he was even younger, and while he was playing chess online. But he’d beaten Carlsen fairly, he insisted.

For weeks now, chess experts have been trying to assess that claim, posting what they’ve found on social media. Some pored over records of Niemann’s games and fed them into chess engines to see how often his moves matched up with those a computer would have made. One expert found that Niemann had played long strings of AI-recommended moves in tournament games. Another analysis agreed that Niemann’s tactics were suspiciously similar to those of a computer, noting that, since the start of 2020, he’s played nine entire games exactly as an engine would have played them; someone else followed up and found that the world-champion Carlsen had managed only two such perfect games during the same stretch. Still others claimed that Niemann’s play was far less computerlike than Carlsen’s. After every seemingly damning analysis, critics denounced the findings and methodology. Finally, on October 4, the online platform chess.com published a 72-page report alleging that Niemann had cheated in more than 100 online games, and that his play in six in-person tournaments might warrant further investigation. But even this report did not draw conclusions about the game in which he’d beaten Carlsen.

And so, somewhat shockingly, nearly two months after news of the Carlsen-Niemann affair first broke, the underlying facts remain in doubt. Indeed, one mystery has spawned another: We don’t know whether Niemann cheated—but also, more important, we don’t know whether we could ever know. In an era when chess engines entirely dominate the game—when they’ve so monopolized human players’ thinking, strategy, and preparation that chess has, in certain ways, come to resemble poker—is it really possible to disentangle creativity from computing?

Read: Chess is just poker now

Many of the attempts to find hard evidence of fraud in Niemann’s record have relied on the same, straightforward notion: If his moves were too similar to those of an engine, he probably cheated. In practice, though, the gap between human and computer play widens and narrows in unpredictable ways. For example, sometimes a player will be in a position where a given move is clearly necessary to avoid losing, or capitalizes on an opponent’s obvious blunder. In such cases, the best choice will be just as apparent to any competent player as it would be to a machine. More fundamental, no single, gold-standard chess engine can be used to reliably measure a grandmaster’s play. Different engines prefer different moves, and the same program will change its suggestions depending on how many steps ahead you ask it to calculate.

Setting aside the problems outlined above, it’s possible that even those of moderate skill might play with inhuman-seeming precision for a period of time, just by luck. And an expert player such as Niemann would not need to take hints from an engine over an entire game to get an edge. A careful cheater—one who means to cover their tracks—might instead rely on a couple of well-timed hints. A tip at just “one critical position could be sufficient to make the difference between winning or losing,” the grandmaster Susan Polgar told me.

The method used in the chess.com report tries to get around these problems. It starts in the traditional way, comparing a player’s moves with those of an engine to determine a “strength score.” (The exact details of that calculation are opaque.) If that score is suspiciously high, the site brings in human chess experts to do a move-by-move analysis that is meant to distinguish between reasonably good and impossibly good play. The website’s “chief chess officer” boasted of having used a very large number of online games to design a sort of “DNA crime scene analysis for every chess player in the world.”

Another approach comes from the computer scientist Kenneth Regan, whose statistical method for retroactive cheat detection has been endorsed by the International Chess Federation and the U.S. Chess Federation. (Regan told me that the armchair analyses of Niemann’s play that have been posted on social media are “not scientific.”) At its most basic level, Regan’s method first uses a player’s Elo rating to predict the likelihood that they will make various moves, then compares their actual moves with what a chess AI would have played. Consistent and unlikely engine-quality play above a certain statistical threshold, especially at crucial positions, could indicate misconduct. This approach has not turned up any evidence that Niemann cheated in over-the-board chess. Yet some critics say the method could miss sparing, strategic use of engine assistance. (Regan has thoughts about that.)

Read: We’re witnessing the birth of a new artistic medium

For all their differences, the methods described above all face a common challenge: They have to find a line between human and computer chess that becomes blurrier by the day. Since the mid-2000s, when the skill of chess engines leapt past humans’, computers have become essential to elite preparation: Players must memorize long sequences of moves that a computer deems optimal. And whereas grandmasters used to trust their intuition and create elaborate attacks, treating the game almost like an art, now many look to engines to generate new strategies. “The creative impulse is coming very strongly now from the engines,” Matthew Sadler, a grandmaster and expert on chess AI, told me.

The young generation of chess players—comprising those who came of age in the era of engine dominance—is likely to be a product of these digital impulses. Indeed, Niemann has claimed that he spent the past two years shut indoors, studying chess with limited human interaction. Polgar, who is 53 years old and learned to play without computers, agreed that players of Niemann’s age and younger are playing “more and more like an engine.” Computers have made previously ignored strategies essential, Sadler said, and changed how, and to what extent, players calculate positions. Given that degree of influence, even a cheat-detection expert like Regan acknowledges that trying to discern whether one or two moves a game came directly from a computer is a tremendous challenge. (All of the chess experts I spoke with insisted that engines’ consistency distinguishes them from humans, but that wouldn’t necessarily allow for the detection of crafty, intermittent cheating.)

Chess is flipping our intuitions about creativity and automation inside out: Computers don’t just execute ideas but conceive them, such that to describe a human player as “machinelike” doesn’t only mean they exhibit great speed and accuracy, like a calculator—the adjective also implies a qualitative change in how they think. Whereas the world champion Garry Kasparov once accused the IBM supercomputer Deep Blue of using human assistance to beat him, the claims against Niemann run in the opposite direction, inverting a decades-old narrative of “artificial” computers mimicking the natural (and by implication superior) intelligence of humans. And it’s not just happening in chess. Humans are starting to take their cues from machines in many other creative endeavors: Grammarly assists writers, DALL-E 2 makes art, AI writes code, programs design clothing. Perhaps one day we won’t compare natural-language programs with Didion or music-composing software with Bach, but the other way around.

Still, the human imagination always seems to find a way of getting out of check. Stone tools, agriculture, writing, the smartphone, and other inventions have been transforming how we think about and interact with the world for millennia, such that “all of the history of artifacts is a history of human intelligence,” Robert Clowes, a philosopher of technology and the mind, told me. Writing did not destroy memory and cognition, as Socrates feared, but instead helped trigger an explosion of art, record keeping, and complex social organization. Similarly, the creative applications of AI might enhance human creativity, allowing us to see the world in unexpected ways—and to develop new genres of music, new understandings of quantum physics, and fields of art and knowledge that are currently unimaginable. “If the new technology we are creating was going to somehow inhibit human intelligence,” Clowes said, “that would be a historical turnaround.”

Some chess experts, at least, are thrilled by what’s to come. Sadler told me that he thinks engines will enable future human players to reach unprecedented levels of skill. Maybe Niemann, the phenom of the first computer-trained generation, is now on track to become the best chess player ever. And maybe chess, a small and nerdy subdomain of human culture, happens to be at the vanguard of a bigger shift—from the time when computers could merely improve cognition to the time when they can change it.