In one of those games, Nieman was losing by 1.3 points in the first 10 moves despite being 100% according to this analysis so I'll take it with a grain of salt. This only looks at having moves within the top 3 engine moves done by one of the engines tested, and sometimes there's just 1-2 good moves so doing 1 out of 10 (or whatever) possible moves doesn't mean you did anything good. Further, it's unclear how cherry-picked it is. If it was that obvious I'd think the other analysis would've caught it which they didn't.
You can find more discussion of it on reddit, but the threads are generally all over the place.
A 1.3 difference out of an opening sideline is neither rare nor lost, and mostly simply comes down to the fact that the engine doesn't understand the opening. Even the mod pin in the link you posted clearly outlines that this is a misleading way to frame this. It would be better to read more into the discussion before helping misinformation spread.
I didn't describe it as lost like the poster did, I said 'losing by 1.3' which is accurate. At any rate, if you actually did read deeper you'd see that losing by 1.3 is plenty relevant, when claiming 100% engine play correlation, and that the mod is somewhat cherry-picking. Further, playing openings where you are -1.3 in 2000 blitz is fine, but in super GM games 1.3 points down is more often than not pretty bad.
"The engine doesn't understand the opening" might have applied 10 years ago but you'll have a very hard time finding a single opening in all of chess where it would be the case now.
While interesting, this does not seem very convincing to me. They successfully show that Niemann was playing many games with a high percentage of "engine perfect" moves, but they do not do enough to show that this is inconsistent with what top players usually do. A whole distribution of scores is shown for Niemann but only limited summary statistics are shown for other top players. A proper comparison would involve showing the same type of data for both.
> They successfully show that Niemann was playing many games with a high percentage of "engine perfect" moves, but they do not do enough to show that this is inconsistent with what top players usually do.
I thought the video very much did make that case. A single known cheating game had a 98% correlation (Sebastien Feller Paris 2010), other GMs have generally at most 75% average correlation. The analysis had more than half a dozen games with Niemann at 100% correlation. If that's cherry picking, it seems like there are a lot of cherries to pick.
Yes, that's the question that I wish she had tried to answer. What are the chances? Without checking for that pattern by other 2600+ GMs, we don't know the answer.
Yeah, because Carlsen's is only a sample of 426 games
The first 4 are the most interesting, having same sample size of 4000. But across the board players tend to have little distinction between choosing moves between 0.0 & 0.1, except one player
Me, as a 1600 player, have played some 0-0-0 games on Lichess. I didn't cheat. I just play a lot of chess games and during those games, my opponent was really bad, so I had a perfect game (according to the engine).
You're conflating accuracy with engine correlation. Having a perfectly accurate game means you didn't make any moves that caused a centipawn loss. Having 100% engine correlation means you're making the exact moves the engine would make.
I think I have on 3-4 occasions played a game where, after evaluating on chess.com, got a 100% accuracy (which is engine correlation). A couple times were all theory and then blundering a mate in 1, but...
I did have one game where I didn't know the theory except a very vague recollection in the beginning. I actually thought I had blundered in that game and was trying to figure out what I'd do if my opponent made a certain move — they didn't find it, I ended up winning material in a tactic and they resigned — I was in complete shock when it came back 100% accuracy (and I definitely did not see the engine response to the move I was worried about, which was the best move).
I'm only around 1600-1700 on chess.com.
Not taking a position either way on Hans, but I have no doubt he knows far more theory than I do (and I do know some lines 20+ moves deep), and correlating with an engine is not impossible even outside of book.
To repeat what was said above, accuracy is not the same as engine correlation.
Engines often play moves that are counterintuitive and weird, but nonetheless good. This is because they can evaluate large trees of tactics in a way that humans cannot.
If a human finds a natural move that is just as good as the engine move (in terms of evaluation), they are still playing accurately, but they are uncorrelated with the engine. Playing accurately is not a sign of cheating. Playing many engine moves is a sign of cheating.
The engine scores centipawn loss against the perfect move (according to the engine). The engine plays the move with the lowest centipawn loss itself. How are those two different?
So there are "really bad" opponents at the 1600 level, but is it reasonable to think there are "really bad" opponents at the 2600 level? It's a different world up there.
Right, op is making a mistake in thinking that a perfect game against a 1600 is the same as a perfect game against a GM. GMs will intentionally play less perfect moves to head towards complications where they will come out ahead. When I start to bang out 15 moves of theory against a IM/GM they will recognize it and play something I’m not familiar with and just win more quickly.
This is complete garbage. Real statistical analysis has been done, and has been inconclusive so far. Cherry picking games is ridiculous - at the 2800 level, a GM will only deviate from the engine's top moves 0-3 times. It would be expected that an exceptional performance would remain within top engine moves if someone was able to play at that level.
Hans isn't at the 2800 level, but Magnus is almost 2900, and probably has good reason to be suspicious of someone playing way above their rating in tough matches.
Yeah this is interesting … she shows that Hans had many tournaments where he shows record setting move accuracy as measured by correlation to Stockfish 15, and that 6 of these tournaments occurred in a row. She also shows that for those tournaments even by Reagans model, Hans results would be like a 1/70000 chance if legit.
The key point (in the end) seems to be that the odds for a streak like Niemann had are about 1 in 80k. Statistically speaking, I'd say that's a long shot from a smoking gun. Here's a good rundown of a case where a cheater is considered to have been exposed by statistical evidence, but those odds were on a completely different scale, 10^22:
Basically the mistake that is easy to make is that we shouldn't ask: "what is the probability that Hans plays five tournaments like that in a row?", but "what is the probability that someone will play five tournaments like that in a row?". Even if we correct for the fact that there are probably more Minecraft speedruns happening than GM tournaments, odds of 80k just seem a bit too low to call it evidence.
Using an ultra-high ELO chess engine to score each possible move, then reversing through the players moves and seeing how often it would have been a positive move (one that shifts the balance of the game in your favor) - or perfect move (not sure which). It is extremely rare to make 100% perfect moves in a game, let alone a series of games. Typical gameplay for high level chess player doesn't peak over 72-75% for a given series of N games. Niemann has several tournaments over this and several games with 100% perfect moves. The inconsistency is also a concern since he goes from mid-60's to 78/79 in a span of one tournament.
It's also worth pointing out that a player's odds of making the perfect move are inverse to their opponent's ELO: as the level of play rises, finding the right play becomes exponentially harder. The data suggests he's sometimes playing other grandmasters as good as those grandmasters would play a rando on Lichess.
yeah - if I pulled a random willing powerball ticket out of my massive pile of powerball tickets, that would be a really, really rare event. It would make me believe that it isn't such a rare thing, for sure.
FM Yosha puts forward a fairly convincing argument about odds and engine correlation, but another commenter rightly pointed out that these statistics are not seen as incriminating in and of themselves. Unfortunately, even when the preponderance of evidence seems to be against a player - best example is Sebastian Feller (https://en.wikipedia.org/wiki/S%C3%A9bastien_Feller) playing with superhuman accuracy at crucial moments, and whose team captain later admitted to helping him cheat - they can still cast enough doubt to be allowed to continue playing at the highest level.
Here is a blunder that Feller played on move 13 just over a month ago (https://new.chess24.com/wall/news/grandmaster-blunders-mate-...) - this same guy managed to draw against Magnus Carlsen in 2008, in a game where Carlsen also found the moves/mannerisms of his opponent highly unusual.
Perfect when compared to the moves the top chess engines would make? Hikaru says he only scored 100% one time, and 70% is more typical for a GM, yet everyone does it?
the gist seems to be that he has unrealistically high correlation with game-engine recommendations, often all the way up to 100%, but only when playing "tough" opponents, and far lower / realistic correlation scores (around 50%) in other games.
for reference, magnus carlsen's correlation score at his peak averages around 70% (according to the video)
This is super cool! I tried it on a video I recorded a while ago that I completely forgot about and was like wait wow the summary came away with points that I'd be glad a viewer got.
One thing is it was a very long and rambling video and probably didn't do a great job of motivating examples rather than just getting bogged down in them for a while, so the summary doesn't really say how the examples support the central claim, but that may be the fault of the video honestly lol...
Also a few basic errors like writing "medium" where I'm pretty sure I said or at least meant "median" and in one case, I'd have to go back and watch this to be sure, but it seems like the summary says something is better in B than in A when I was saying it's better in A than B. The summary definitely touches the right content but I'm not sure it's correct.
Also funnily I have a tendency to sprinkle the word "like" liberally(for better or worse) and the summary copies some of the sentences verbatim, starting with "Like..."
(completely off the topic of cheating in chess, sorry...)
He played several games with 100% correlation with what chess engines considered to be the best move, and also played in 5 consecutive tournaments with such a high fraction of engine-preferred moves that his performance rivals the best players in history at the pinnacle of their careers.
It would be nice to get an ELI5 on this too. I used to play chess and have an understanding of the significance... but I don't think I can fully appreciate it as well as someone with a solid background in both.
It's actually fairly easy to cheat statistics. It happens literally all the time in Academia. There's a thousand ways to make a statistical analysis believably say what you want in a way where even other professionals don't realize is the case unless an expert does a thorough analysis.
It's easy to cheat a statistic you create. It's quite hard to cheat a statistic where you don't know who will look when at which particular data points.
I did not intend to attack or defend Hans with that statement, I just wanted to point out that both you and the original comment could be right at the same time. That being said, it's quite funny that this case showed both sides.
Since younger players are growing up and developing with these incredibly powerful chess engines, I wonder if that plays a role in their ability to play at such a level.
If this were the case, I think we'd see younger players more likely to get these 100s more often as they're learning from chess engines.
Does anyone familiarized with the topic know if this makes sense?
You can find more discussion of it on reddit, but the threads are generally all over the place.
https://www.reddit.com/r/chess/comments/xofl99/one_of_the_10...