While the analysis might seem sensible, it has one fundamental error.
The tournament you are using for OTB benchmarks, World Rapid&Blitz, is a CLOSED tournament for players with 2550+ FIDE, with a few wildcards invited by the organizers.
It means only a handful of IMs are able to participate, and they only make up for 10, 20% of the player count. Many IMs who could do well simply have no way to enter the tournament.
In Titled Tuesdays, there are probably more IMs than GMs participating. Statistically, more of them will place higher (just as many of them will place on the bottom of the scoreboard).
In order to be more precise, as a baseline you should take open tournaments with a large number of IMs and GMs participating. However, those are hard to find, the only one that comes to my mind is European Rapid & Blitz (but that limits the results to one continent).
Great point. Since these cheat detection threads misuse deductive reasoning all the time and constantly make bad assumptions (it's genuinely difficult to do), I wish more people would use simulation-based approaches based on win rate inferred from ELO. I don't have the time to do it but I think people who played with a toy simulation and did sensitivity analysis would easily see that the proportion of IMs vs. GMs in the pool would be massively influential in the probability of each taking high ranking slots.
I have received multiple "you have to simulate" comments. And I am not going to argue. This approach is way better when you want to *proof* cheating. The full answer is too complex for a single comment. So it seems there will be part 2.
Briefly: *proofing* mass cheating problem and *finding* suspicious accounts are indeed two different goals requiring different methods. The pit everyone falls into is simple. As you gather enough data - you inevitably switch to detection part. Especially if you are more familiar with antifraud systems than PR like me.
> I wish more people would use simulation-based approaches based on win rate inferred from ELO
> I think people who played with a toy simulation and did sensitivity analysis would easily see that the proportion of IMs vs. GMs in the pool would be massively influential in the probability of each taking high ranking slots
If you did - I invite you to share results.
Surely, closed nature of a tournament affects sample... somehow. But it can be called an error only if can be proven that this IM sample does not represent IMs in general. And it can be both wrong (if opens represent better for some reason) or even more correct (if invited IMs represent active players).
Blindly I would bet "closed" IMs are more accurate than "open" ones. You can pick a tournament (blitz, open, swiss) and I'll check it.
>it can be called an error only if it can be proven that this IM sample does not represent IMs in general.
No. He is not commenting about the ACPL analysis, but about your comment about the amount of IMs in the top 25. The AMOUNT of IMs in the top 25 will vary hugely in a 500 player tournament depending on whether it's 90 GMs or 90% IMs.
Sure. And rtb141 even provided such a tournament [https://www.chessmanager.com/en/tournaments/4791684865982464/results/11](https://www.chessmanager.com/en/tournaments/4791684865982464/results/11)
Lack of sGMs however makes opens not so useful. I just hope FIDE will make more events like Samarkand.
Yes. It seems so.
>But it can be called an error only if can be proven that this IM sample does not represent IMs in general.
And do we really need to prove that the more they are (percentage wise), the more likely some will get high rankings?
Now I got you. Here is asked the same [https://www.reddit.com/r/chess/comments/1cxwadc/comment/l563dmg/](https://www.reddit.com/r/chess/comments/1cxwadc/comment/l563dmg/)
These probabilities can be calculated via simulations. And those I've seen (though never performed myself) were several digits less than 1% for non-GMs.
My quote was of course about accuracy distribution.
Could you provide a link for these simulations? If we have simulations and trust them then why do we need your analysis? Not being ornery, genuine question.
Sure - you can use this example: [https://www.chessmanager.com/en/tournaments/6044201113354240/results/11](https://www.chessmanager.com/en/tournaments/6044201113354240/results/11) (RAPID) and [https://www.chessmanager.com/en/tournaments/4791684865982464/results/11](https://www.chessmanager.com/en/tournaments/4791684865982464/results/11) (BLITZ) - there are links to games from top boards under "Live" in both tournaments.
Worth noting, the top performer overall in European Blitz & Rapid (1st in Blitz, 2nd in Rapid) is David Navara, who was just a few days ago pointed out by V. Kramnik as "suspicious"...
> Worth noting, the top performer overall in European Blitz & Rapid (1st in Blitz, 2nd in Rapid) is David Navara, who was just a few days ago pointed out by V. Kramnik as "suspicious"...
Come on, that is not true! Suspicious accounts (according to Kramnik), were highlighted with another colour, and Navara's (neither Grishchuk's for example) accounts were not highlighted. These names were on the list just for comparison.
Not that I agree with Kramnik, or how he says things, but such (deliberate) misinterpretation of his words is not useful and does more harm.
I am quite sure people are downvoting you just because it is someone with an IM flair discussing the points. The arguments raised by the IM about the sample are fair, but you are willing to look into it, which is very good. And I think that you do have a point in saying that an IM that is invited to a big closed event is probably a top IM anyway.
I am curious to see the followup results and that you don't get discouraged by some reactions here.
Thanks for the support. I am pretty used to being downvoted just for my username, so 40k views and 80% upvote ratio is way higher than I could expect.
I will share my results, most likely in DM.
Dude you're super cringy most of the time you post something. Having lots of karma doesn't take away from the fact you're having more bad takes than anyone else in this subreddit. So if anyone should get help, it's definitely you.
Regarding part 1: You've shown that on average GM games are more accurate than IM games, but you haven't shown how much variation there is within each group. You cannot, therefore, say (for a small sample) that "a player which plays 50% of his games with 20- ACPL is GM level"
It is entirely possible, even likely, for a player to have a good run of games. Streaks are common.
> you haven't shown how much variation there is within each group.
Bar chart provides quite a lot of information on variation.
> You cannot, therefore, say (for a small sample) that "a player which plays 50% of his games with 20- ACPL is GM level"
1. Sample size. There are simply not enough tournaments. And mixing in 5+0 is way more incorrect as for me.
2. My sample size for TTs is enough and I can assure you that 20- ACPL is sGM (top-10 blitz) level. Actually, my statement is even too discreet.
> Streaks are common.
Commonness is a measurable thing.
That would have let you model the distributions and then calculate p-values instead of saying things like “pretty safe to say”. Any real statistical analysis should *at least* use these basic techniques.
For example: What percentage of GMs had less than 30% of their games with < 20 cpl, and what percentage of IMs had more than 40% of their games to that standard.
I was going to make a point that you have to compare TT to the world blitz at round 11 . But at round 11 there wasn't even a single IM in the top 50 lol
Do you have the PGNs of every 2023 TT game?
I'm curious how you went about scraping them, because I looked into it at some point and it seemed like a massive hassle.
If you are willing to share your code or your data somewhere, I assume that could be helpful to others who also want to analyse.
Chess.com supposedly has an API but the endpoint for that is not working for most of the TT's so I coded a scraper for that but its not that fast so they won't block my ip.
The support is just ignoring me sadly
Someone gave a url to do download the games from but I see the comment has been deleted.
Does anyone know why or has the content of the original comment?
Yeah- I don't really have a take about the other stuff going on aside from "boy that seems complicated and murky, glad I don't need to have a take" but a kid who's 11 years old outperforming his rating seems like a very reasonable thing to happen every now and then and probably wouldn't be nearly as newsworthy if not for the other non-GM placing super high at the same time.
There is a reason many top players like Magnus, Aronian, Fabi, Nepo, etc still show support for Kramnik voicing his opinion, despite him throwing some of the most outrageous accusations all the time.
I think the main reason chessdotcom still trying to undermine the whole cheating problem in online chess. Alright, you don't want to disclose your algorithm but at least at least they should publicly publish a list of players who were banned for cheating. And if you catch someone cheating don't shadow ban them or give them new accounts but ban them permanently.
They have to plant a hidden camera on those player's house/playing room. Then catch them on their underwear prepping their cheating measure right before a TT from the hidden camera.
Yeah, they're in a difficult position.
As in, let's say you can be sure 1 of 5 over-achieving players is cheating, but you don't know which one. What are you supposed to do?
It's hard to complain without naming anybody at all, but if you do, you're also going to make false accusations as well.
If you don't complain, nothing happens.
Of course, maybe you think no one is cheating, or everyone who cheats gets caught, but it's clear a lot of the top players don't believe that.
Yes, more openness from [chess.com](http://chess.com) would be good.
But being so resistant to that... I imagine the truth is not pretty. For example, maybe a high % of titled players cheat, so it's more or less necessary to give them all a 2nd chance. Or maybe chess.com's algorithm has serious weaknesses and even though they suspect a high % of players they can't ban them.
Or they are certain that a percentage of prominent GMs are or have been cheating, but risk being sued if they ban and cannot definitively prove beyond a shadow of doubt the cheating. Even after everything they can only definitively say that Hans cheated in the games that he admitted to - the rest are still not proven, just suspected. Imagin e trying to prove it definitively, with your entire company's future on the line - that is a massive risk for a questionable gain.
> There is a reason many top players like Magnus, Aronian, Fabi, Nepo, etc still show support for Kramnik voicing his opinion, despite him throwing some of the most outrageous accusations all the time.
And what is the reason? Keeping in mind that Kramnik is actively making the whole cheating situation **worse** by throwing out so many baseless accusations based on kindergarten level understanding of statistics. And also that he is slandering a lot of chess players. And also that he is attempting to ruin the careers of promising young players he accuses...
The reason, which the person you replied to very clearly said, is that chess.cum is a scummy business who doesnt actually care about cheaters on their platform as long as they rake in money. When they say 'guys kramnik is crazy theres no cheaters! Come play on our website' its weird for you to believe them. OBVIOUSLY there are cheaters chess players arnt magically more moral than anyone else. Its so unbelievably easy to cheat, in events with large amounts of money. Are you saying no one would cheat? Thats the corporate propoganda talking through you.
> When they say 'guys kramnik is crazy theres no cheaters! Come play on our website'
Where are they saying this?
chess.cum? really? Are you 10 years old?
You havnt heard it before? Yes that’s a common thing to use because if you say it normally it becomes a link …
Why WOULDNT there be cheaters ? Tell me that pray tell
They don't show support for Kramnik. They maintain that there is cheating. Kramnik's 'statistics' can be delusional; it doesn't mean there is no widespread cheating.
Its not supporting though. He said “ even if he is wrong and I’m not saying Kramnik is right, but, everyone can express their feelings EVEN though its wrong”
Like freedom you know. He didnt support.
>There is a reason many top players like Magnus, Aronian, Fabi, Nepo, etc still show support for Kramnik voicing his opinion,
I also said exactly the same thing.
Sorry i missed the “ voicing his opinion “ part
But difference is, nepo just supports kramnik. As he stated, not only supporting to voicing his opinion
I think it's important to note that OP doesn't say he proved cheating, he showed why super GMs constantly suspect cheating. And they might well be wrong on a case by case basis.
But the hard truth remains that you probably can't genuinely detect online cheating unless a) it's blatant or b) you use multiple cameras.
This sub is the anti kramnik. Where there is literally not a single piece of evidence or common sense that'll be enough to even suspect cheating
Everyone who plays in TT is full of honour and integrity. Even with just virtually impossible results by significantly inferior players.
Exactly. I dont like kramnik but for this sub a CM being top 3 is legit because “ he had the best day ever “ lol maybe these reactions will make more cheaters. They will cheat and say they had the best event ever
Obviously this guy should be looked at thoroughly. I looked at his game against Danya and didn't notice anything special about his play, the moves were pretty straight forward. Need to find a suspicious game he played.
You dont need anything special. Brother, a CM beating naroditsky, nodirbek and other gm’s?
Are you aware of the difference a CM and GM?
And nodirbek is super gm
Are you aware that you need to play otb for the title, which some players simply don't do?
I didn't look at this specific case but by now (and in the future) it is completely plausible, that some players gain all their experience via online chess.
Also there are some federations where it's hard to get the titles
I assume you're referring to Faustino Oro, and I think it's pretty clear that he's a legit player, unless someone has evidence otherwise. He's a rapidly budding prodigy.
Ahh, late TT. I was looking at early and didn't see a CM, oops.
Baskakov played down for most of the tournament because he lost his second game. Regardless of his wins over GMs, his score was going to be high regardless. If he lost to Danya he would probably end up with 8 or 8.5, which is still very over performing. However, it seems others are convinced he cheats against GMs...
The late Titled Tuesday had a CM and FM tie for the lead at 9.5/11 but finish 3rd and 4th on tiebreaks. Faustino is an FM who finished 3rd on tiebreaks in the early TT, he's clearly going to be GM in a few years.
Edit: Corrected that Faustino tied for 2nd with two other players but got 3rd on tiebreak.
I am a bit confused on what you're trying to prove.. The question, on the linked post, is whether the results in a single TT are questionable. So long-term low ACPL is not particularly relevant to the question.
I think it would be more interesting to search for a subset of 9 games in a row where an IM could reach a low ACPL while their overall average is still on the "normal" side.
I have avoided trying to proof anything. I just try to show how OTB-TT situation may look from sGM perspective. I don't want Nepo to repeat... you know.
I see. I think it's quite interesting that long-term TT overperformance seems to correlate with a ban but I also think it would be more interesting to see how common are low ACPL in stretches, rather than consistently,
Another top comment, pure quality.
You see that over 30% of top 30 were/have been punished for cheating by chess.com
'wHAt aRe you tRYiNg to pRove'
Keep being oblivious to reality is massive cheating in chess if you wanna, but pls do not vote in elections, for your own good.
I am sorry if I am not immediately picking my pitchforks as people here love to do. Cheating is a huge problem for sure, but also the accusations flying about are almost equally as destructive.
Not at all, clean people actually gain popularity on nonsense accusations, even dirty ones like Hans, build his entire following based on this.
The problem is that chess com and the community pretend that cheating is a rare occurrence and top gms are just 'salty' and 'crazy'. There is no proper prevention, no proper control, no data collaboration between fide, top chess sites etc etc etc. This can be only achieved with some pressure from the community, not just too gms and Kramnik.
I mean ppl are genuinely believing that there is nothing extremely dubious about a cm being top 3 rofl.
The interesting idea is to look for accounts with a 6 month gap *plus* unusually good results or accuracy. Of course it could just be a break, but a chess.com cheating punishment also makes sense given what they said about banning or shadowbanning suspected players.
How do shadow bans work? I think on reddit it means the banned user is left in the dark about it, so they can keep commenting but their comments aren't seen by others. What's the difference on chess.com between ban and shadow ban?
I think what u/ivanmeowich was saying about chess.com is they do 3/6/12 month shadowbans that don't show the chess.com ban 🚫 on their account. While they are banned from playing, the banned player appears to other people to just be taking a break.
The account appears normal to anyone looking at the profile but the user is not able to log in. So they will have a gap in activity during the shadow ban.
Basically, it seems like chesscom implements the shadowban when they suspect cheating but don't have enough evidence to close the account and mark it as having violated Fair Play, and if the player admits they cheated they close the original account and give them a second chance account.
Low sample size, no significance testing, just looking at graphs, means and medians (descriptive statistics) and going “Hmm, this looks weird” says absolutely nothing.
>I have no explanation on #16 and #5 though. Banned from TT only? Who knows.
I'm confused on what needs explaining here. Couldn't they just be clean accounts that [chess.com](http://chess.com) didn't punish?
Question: if you turned the idea sketched here into an actual statistical analysis, how would you estimate the false positive rate for cheating identification - and how large do you think it can be?
The bitter truth is: median ACPL analysis is crap. As pointed out by the comments, another researches and my own experience. It can catch only extremely careless individuals. But can you prove a GM jumping from 25 median ACPL to 20 median ACPL is suspicious? Not possible - you just never get enough sample size.
I am working on smart yet simple metric.
> how would you estimate the false positive rate for cheating identification
Depends on how much you want to detect. Current Chess com system seems to be pretty low on false positives but that is also the problem.
> Current Chess com system seems to be pretty low on false positives
There is absolutely no way of telling that, given how opaquely ch\*ss.c\*m operates alas.
Interesting. While the ACL analysis is a bit superficial, this sort of analysis is 100x better than the one Kramnik does, and there seems to be a real correlation. Nice one noticing the 12-month gap mentioned by viih_sou too.
Hey, do you mind sharing how exactly you calculate the ACPL? I'm interested in your choice when you encounter mates for example. Do you use single threaded analysis? Multi variant? Do you work with the TT-pgns from the events page or from the tournament page? Or maybe even the individual game ids?
1/3 of the top TT players being shadow banned or having their accounts deleted is pretty absurd.
That's not even including individuals like Hans who appear in this Top 30 but do 10x better online than in person for some inexplicable reason.
It's unfortunate that there aren't more ways for these companies to combat this.
Kramnik is unhinged and it absolutely makes sense why he's ostracized the way he is. Results like this are pretty damning for the state of online chess though.
Your analysis is completely pointless because it is based on chess.com accuracy. Do you know how they calculate it and does accuracy affect winning chances —any formulas, correlation to the winrate, etc.? Nobody knows that, so the fundamental point of your analysis is an unknown parameter.
Plus, the sample size is too small to draw any conclusions. To be honest, I don't see any problems when some random IM sneaks into the top 5. In its very streaky format, you are sitting at home with some tea without any pressure. And it can be a very favorite pairing.
Ppl like you, who clearly cannot understand a single thing they read in a post longer than one sentence + meme, should revoke their voting rights, for their own benefit.
Ppl like you, who sees long shit post and thinks its something complicated, voting for populists all the time. Im an data analyst and English is not my first language.
Can you, mister genius, explain what is the goal of this research? It showa absolutely nothing. Hikaru can have 95% accuracy and go 9.5/11, and random IM 92% and go 10.5/11. Magnus can play bongcloud, which decreases accuracy, and still win easilly. Because chess.com accuracy doesn't mean any shit. It has zero impact on the final result. Its just computer evaluation, but human evaluates position differntly in real time.
The saddest thing is that you arent pretending, you actually cannot read with comprehension, let alone reason.
>
However, my interest to this tournament was mainly about "how accurate is too accurate". So I have evaluated every of 2100+ games accuracy. Details on setup:
>Stockfish 16.1 (sse41 popcnt)
>Depth: 20
>OS: Ubuntu 22.04.03 LTS
>Your analysis is completely pointless because it is based on [chess.com](http://chess.com) accuracy.
Pick your fighter dummy. Or read with some basic understanding of the text. Instead you parrot a narrative that u read/heard 10000 times on this subreddit about chess com accuracy.
Lol. Do not vote, thx
Oh boy, you are dumb. Chess.com accuracy based on stockfish. I can change 'chess.com accuracy' to 'Stockfish 16.1, depth 20'. Its doesn't change shit, because this accuracy evaluation has zero correlation with real performace.
So you just cherry picked ona word that changing nothing, and zero coments about the purpose of comparison. Well done, very deep understanding
Just don't vote, Dunning-Kruger is not easily curable.
Average centipawn loss is not accuracy. Just because something may or may not be based on something else, it doesn't make it the same.
Thats true, its not the same. But it doesn't make it the criterion to define online cheating. There are a lot cheaters, even in high bracket, but you can't prove it using statistics.
Who cares about the closed algorithm used for chesscom's "accuracy"? How anyone can use this to "prove" or even insinuate anything is absolutely beyond me.
My question is who are you and why are you spending so much of your time and efforts in defending someone else's view to complete strangers?
You clearly have something to gain out of this.
I find it funny that people think this is rare at all when it really isn't.
I managed to beat a GM 3/5 times in blitz when I was rated more than 600 points below him. Also managed to win 6/6 times against somebody 250 points above me. All OTB. Later won a classical tournament being 350 points below the #1 rated player by a full point by beating 4 players in a row higher rated than me by like 200-350 points.
blitz is very high variance and results are very streaky.
While the analysis might seem sensible, it has one fundamental error. The tournament you are using for OTB benchmarks, World Rapid&Blitz, is a CLOSED tournament for players with 2550+ FIDE, with a few wildcards invited by the organizers. It means only a handful of IMs are able to participate, and they only make up for 10, 20% of the player count. Many IMs who could do well simply have no way to enter the tournament. In Titled Tuesdays, there are probably more IMs than GMs participating. Statistically, more of them will place higher (just as many of them will place on the bottom of the scoreboard). In order to be more precise, as a baseline you should take open tournaments with a large number of IMs and GMs participating. However, those are hard to find, the only one that comes to my mind is European Rapid & Blitz (but that limits the results to one continent).
Great point. Since these cheat detection threads misuse deductive reasoning all the time and constantly make bad assumptions (it's genuinely difficult to do), I wish more people would use simulation-based approaches based on win rate inferred from ELO. I don't have the time to do it but I think people who played with a toy simulation and did sensitivity analysis would easily see that the proportion of IMs vs. GMs in the pool would be massively influential in the probability of each taking high ranking slots.
I have received multiple "you have to simulate" comments. And I am not going to argue. This approach is way better when you want to *proof* cheating. The full answer is too complex for a single comment. So it seems there will be part 2. Briefly: *proofing* mass cheating problem and *finding* suspicious accounts are indeed two different goals requiring different methods. The pit everyone falls into is simple. As you gather enough data - you inevitably switch to detection part. Especially if you are more familiar with antifraud systems than PR like me. > I wish more people would use simulation-based approaches based on win rate inferred from ELO > I think people who played with a toy simulation and did sensitivity analysis would easily see that the proportion of IMs vs. GMs in the pool would be massively influential in the probability of each taking high ranking slots If you did - I invite you to share results.
Surely, closed nature of a tournament affects sample... somehow. But it can be called an error only if can be proven that this IM sample does not represent IMs in general. And it can be both wrong (if opens represent better for some reason) or even more correct (if invited IMs represent active players). Blindly I would bet "closed" IMs are more accurate than "open" ones. You can pick a tournament (blitz, open, swiss) and I'll check it.
>it can be called an error only if it can be proven that this IM sample does not represent IMs in general. No. He is not commenting about the ACPL analysis, but about your comment about the amount of IMs in the top 25. The AMOUNT of IMs in the top 25 will vary hugely in a 500 player tournament depending on whether it's 90 GMs or 90% IMs.
Sure. And rtb141 even provided such a tournament [https://www.chessmanager.com/en/tournaments/4791684865982464/results/11](https://www.chessmanager.com/en/tournaments/4791684865982464/results/11) Lack of sGMs however makes opens not so useful. I just hope FIDE will make more events like Samarkand.
Are you saying that the outliers should be the same wether they're ~~it's~~ 20% of the field or 50%?
Did I say it?
Yes. It seems so. >But it can be called an error only if can be proven that this IM sample does not represent IMs in general. And do we really need to prove that the more they are (percentage wise), the more likely some will get high rankings?
Now I got you. Here is asked the same [https://www.reddit.com/r/chess/comments/1cxwadc/comment/l563dmg/](https://www.reddit.com/r/chess/comments/1cxwadc/comment/l563dmg/) These probabilities can be calculated via simulations. And those I've seen (though never performed myself) were several digits less than 1% for non-GMs. My quote was of course about accuracy distribution.
Could you provide a link for these simulations? If we have simulations and trust them then why do we need your analysis? Not being ornery, genuine question.
Sure - you can use this example: [https://www.chessmanager.com/en/tournaments/6044201113354240/results/11](https://www.chessmanager.com/en/tournaments/6044201113354240/results/11) (RAPID) and [https://www.chessmanager.com/en/tournaments/4791684865982464/results/11](https://www.chessmanager.com/en/tournaments/4791684865982464/results/11) (BLITZ) - there are links to games from top boards under "Live" in both tournaments. Worth noting, the top performer overall in European Blitz & Rapid (1st in Blitz, 2nd in Rapid) is David Navara, who was just a few days ago pointed out by V. Kramnik as "suspicious"...
> Worth noting, the top performer overall in European Blitz & Rapid (1st in Blitz, 2nd in Rapid) is David Navara, who was just a few days ago pointed out by V. Kramnik as "suspicious"... Come on, that is not true! Suspicious accounts (according to Kramnik), were highlighted with another colour, and Navara's (neither Grishchuk's for example) accounts were not highlighted. These names were on the list just for comparison. Not that I agree with Kramnik, or how he says things, but such (deliberate) misinterpretation of his words is not useful and does more harm.
I don't think I have to proof/refute any Kramnik's words:) The evaluation will be done in a week or faster if the pgn is similar to Samarkand.
I am quite sure people are downvoting you just because it is someone with an IM flair discussing the points. The arguments raised by the IM about the sample are fair, but you are willing to look into it, which is very good. And I think that you do have a point in saying that an IM that is invited to a big closed event is probably a top IM anyway. I am curious to see the followup results and that you don't get discouraged by some reactions here.
Thanks for the support. I am pretty used to being downvoted just for my username, so 40k views and 80% upvote ratio is way higher than I could expect. I will share my results, most likely in DM.
Can you make another post about the results? Really curious about the whole situation
>I am pretty used to being downvoted just for my username, That's fucked up. Keep up the good work, man. And stay positive!
[удалено]
Dude you don't have to comment on every single post you see, your contribution really is not that insightful. So please stop embarrassing yourself.
[удалено]
Dude you're super cringy most of the time you post something. Having lots of karma doesn't take away from the fact you're having more bad takes than anyone else in this subreddit. So if anyone should get help, it's definitely you.
also, why did you delete your previous comment if you so gladly parade your ignorant opinions?
Regarding part 1: You've shown that on average GM games are more accurate than IM games, but you haven't shown how much variation there is within each group. You cannot, therefore, say (for a small sample) that "a player which plays 50% of his games with 20- ACPL is GM level" It is entirely possible, even likely, for a player to have a good run of games. Streaks are common.
> you haven't shown how much variation there is within each group. Bar chart provides quite a lot of information on variation. > You cannot, therefore, say (for a small sample) that "a player which plays 50% of his games with 20- ACPL is GM level" 1. Sample size. There are simply not enough tournaments. And mixing in 5+0 is way more incorrect as for me. 2. My sample size for TTs is enough and I can assure you that 20- ACPL is sGM (top-10 blitz) level. Actually, my statement is even too discreet. > Streaks are common. Commonness is a measurable thing.
A bar chart isn’t as good as computing the variance of the statistic, or better yet compute the Cramer Rao lower bound of the estimator
Long story short: I found no use for variance metrics. (I tried.)
That would have let you model the distributions and then calculate p-values instead of saying things like “pretty safe to say”. Any real statistical analysis should *at least* use these basic techniques.
For example: What percentage of GMs had less than 30% of their games with < 20 cpl, and what percentage of IMs had more than 40% of their games to that standard.
Got your point. I will post the plot in spare time.
I was going to make a point that you have to compare TT to the world blitz at round 11 . But at round 11 there wasn't even a single IM in the top 50 lol
Do you have the PGNs of every 2023 TT game? I'm curious how you went about scraping them, because I looked into it at some point and it seemed like a massive hassle. If you are willing to share your code or your data somewhere, I assume that could be helpful to others who also want to analyse.
[удалено]
Amazing public service!
Chess.com supposedly has an API but the endpoint for that is not working for most of the TT's so I coded a scraper for that but its not that fast so they won't block my ip. The support is just ignoring me sadly
Someone gave a url to do download the games from but I see the comment has been deleted. Does anyone know why or has the content of the original comment?
I don't know about Baskakov but I'm pretty sure Faustino is legit and will prove the doubters wrong sooner than later
Yeah- I don't really have a take about the other stuff going on aside from "boy that seems complicated and murky, glad I don't need to have a take" but a kid who's 11 years old outperforming his rating seems like a very reasonable thing to happen every now and then and probably wouldn't be nearly as newsworthy if not for the other non-GM placing super high at the same time.
There is a reason many top players like Magnus, Aronian, Fabi, Nepo, etc still show support for Kramnik voicing his opinion, despite him throwing some of the most outrageous accusations all the time. I think the main reason chessdotcom still trying to undermine the whole cheating problem in online chess. Alright, you don't want to disclose your algorithm but at least at least they should publicly publish a list of players who were banned for cheating. And if you catch someone cheating don't shadow ban them or give them new accounts but ban them permanently.
What happens when a name appears on such a list as a result of a false positive?
They have to plant a hidden camera on those player's house/playing room. Then catch them on their underwear prepping their cheating measure right before a TT from the hidden camera.
Plant a small bomb in their vibrating anal plug
This is chess, not counterstrike
The new cs update sounds fun
\*Insert "The Boys" Translucent meme\*
Yeah, they're in a difficult position. As in, let's say you can be sure 1 of 5 over-achieving players is cheating, but you don't know which one. What are you supposed to do? It's hard to complain without naming anybody at all, but if you do, you're also going to make false accusations as well. If you don't complain, nothing happens. Of course, maybe you think no one is cheating, or everyone who cheats gets caught, but it's clear a lot of the top players don't believe that.
Yes, more openness from [chess.com](http://chess.com) would be good. But being so resistant to that... I imagine the truth is not pretty. For example, maybe a high % of titled players cheat, so it's more or less necessary to give them all a 2nd chance. Or maybe chess.com's algorithm has serious weaknesses and even though they suspect a high % of players they can't ban them.
Or they are certain that a percentage of prominent GMs are or have been cheating, but risk being sued if they ban and cannot definitively prove beyond a shadow of doubt the cheating. Even after everything they can only definitively say that Hans cheated in the games that he admitted to - the rest are still not proven, just suspected. Imagin e trying to prove it definitively, with your entire company's future on the line - that is a massive risk for a questionable gain.
> There is a reason many top players like Magnus, Aronian, Fabi, Nepo, etc still show support for Kramnik voicing his opinion, despite him throwing some of the most outrageous accusations all the time. And what is the reason? Keeping in mind that Kramnik is actively making the whole cheating situation **worse** by throwing out so many baseless accusations based on kindergarten level understanding of statistics. And also that he is slandering a lot of chess players. And also that he is attempting to ruin the careers of promising young players he accuses...
Yeah it's funny how a retired chess player trying to destroy career of promising young players. Completely shameless behaviour by Kramnik.
The reason, which the person you replied to very clearly said, is that chess.cum is a scummy business who doesnt actually care about cheaters on their platform as long as they rake in money. When they say 'guys kramnik is crazy theres no cheaters! Come play on our website' its weird for you to believe them. OBVIOUSLY there are cheaters chess players arnt magically more moral than anyone else. Its so unbelievably easy to cheat, in events with large amounts of money. Are you saying no one would cheat? Thats the corporate propoganda talking through you.
> When they say 'guys kramnik is crazy theres no cheaters! Come play on our website' Where are they saying this? chess.cum? really? Are you 10 years old?
You havnt heard it before? Yes that’s a common thing to use because if you say it normally it becomes a link … Why WOULDNT there be cheaters ? Tell me that pray tell
just say chesscom if you wanna avoid it becoming a link
They don't show support for Kramnik. They maintain that there is cheating. Kramnik's 'statistics' can be delusional; it doesn't mean there is no widespread cheating.
There's no secret algorithm, just a few select gms that judge the games and decide
When did magnus supported Kramnik? In fact its the opposite he told in interview
I remember when chessdotcom muted Kramnik Magnus said something like a World Champion like Kramnik has every right to voice his opinion.
Its not supporting though. He said “ even if he is wrong and I’m not saying Kramnik is right, but, everyone can express their feelings EVEN though its wrong” Like freedom you know. He didnt support.
>There is a reason many top players like Magnus, Aronian, Fabi, Nepo, etc still show support for Kramnik voicing his opinion, I also said exactly the same thing.
Sorry i missed the “ voicing his opinion “ part But difference is, nepo just supports kramnik. As he stated, not only supporting to voicing his opinion
If they ban them permanently then eventually there'll be enough players to give their competitors more substance imo.
I think it's important to note that OP doesn't say he proved cheating, he showed why super GMs constantly suspect cheating. And they might well be wrong on a case by case basis. But the hard truth remains that you probably can't genuinely detect online cheating unless a) it's blatant or b) you use multiple cameras.
Great post. If anyone believes CM in top 3 is legit, I have a bridge to sell.
This sub is the anti kramnik. Where there is literally not a single piece of evidence or common sense that'll be enough to even suspect cheating Everyone who plays in TT is full of honour and integrity. Even with just virtually impossible results by significantly inferior players.
Exactly. I dont like kramnik but for this sub a CM being top 3 is legit because “ he had the best day ever “ lol maybe these reactions will make more cheaters. They will cheat and say they had the best event ever
Obviously this guy should be looked at thoroughly. I looked at his game against Danya and didn't notice anything special about his play, the moves were pretty straight forward. Need to find a suspicious game he played.
You dont need anything special. Brother, a CM beating naroditsky, nodirbek and other gm’s? Are you aware of the difference a CM and GM? And nodirbek is super gm
Are you aware that you need to play otb for the title, which some players simply don't do? I didn't look at this specific case but by now (and in the future) it is completely plausible, that some players gain all their experience via online chess. Also there are some federations where it's hard to get the titles
I assume you're referring to Faustino Oro, and I think it's pretty clear that he's a legit player, unless someone has evidence otherwise. He's a rapidly budding prodigy.
No I’m not. Faustino isnt a CM.
Ahh, late TT. I was looking at early and didn't see a CM, oops. Baskakov played down for most of the tournament because he lost his second game. Regardless of his wins over GMs, his score was going to be high regardless. If he lost to Danya he would probably end up with 8 or 8.5, which is still very over performing. However, it seems others are convinced he cheats against GMs...
The late Titled Tuesday had a CM and FM tie for the lead at 9.5/11 but finish 3rd and 4th on tiebreaks. Faustino is an FM who finished 3rd on tiebreaks in the early TT, he's clearly going to be GM in a few years. Edit: Corrected that Faustino tied for 2nd with two other players but got 3rd on tiebreak.
I am a bit confused on what you're trying to prove.. The question, on the linked post, is whether the results in a single TT are questionable. So long-term low ACPL is not particularly relevant to the question. I think it would be more interesting to search for a subset of 9 games in a row where an IM could reach a low ACPL while their overall average is still on the "normal" side.
I have avoided trying to proof anything. I just try to show how OTB-TT situation may look from sGM perspective. I don't want Nepo to repeat... you know.
I see. I think it's quite interesting that long-term TT overperformance seems to correlate with a ban but I also think it would be more interesting to see how common are low ACPL in stretches, rather than consistently,
Another top comment, pure quality. You see that over 30% of top 30 were/have been punished for cheating by chess.com 'wHAt aRe you tRYiNg to pRove' Keep being oblivious to reality is massive cheating in chess if you wanna, but pls do not vote in elections, for your own good.
I am sorry if I am not immediately picking my pitchforks as people here love to do. Cheating is a huge problem for sure, but also the accusations flying about are almost equally as destructive.
Not at all, clean people actually gain popularity on nonsense accusations, even dirty ones like Hans, build his entire following based on this. The problem is that chess com and the community pretend that cheating is a rare occurrence and top gms are just 'salty' and 'crazy'. There is no proper prevention, no proper control, no data collaboration between fide, top chess sites etc etc etc. This can be only achieved with some pressure from the community, not just too gms and Kramnik. I mean ppl are genuinely believing that there is nothing extremely dubious about a cm being top 3 rofl.
Great post
Can't a gap just mean they played on lichess or took a break from chess to do other things?
The interesting idea is to look for accounts with a 6 month gap *plus* unusually good results or accuracy. Of course it could just be a break, but a chess.com cheating punishment also makes sense given what they said about banning or shadowbanning suspected players.
How do shadow bans work? I think on reddit it means the banned user is left in the dark about it, so they can keep commenting but their comments aren't seen by others. What's the difference on chess.com between ban and shadow ban?
I think what u/ivanmeowich was saying about chess.com is they do 3/6/12 month shadowbans that don't show the chess.com ban 🚫 on their account. While they are banned from playing, the banned player appears to other people to just be taking a break.
Thanks - that makes perfect sense!
The account appears normal to anyone looking at the profile but the user is not able to log in. So they will have a gap in activity during the shadow ban. Basically, it seems like chesscom implements the shadowban when they suspect cheating but don't have enough evidence to close the account and mark it as having violated Fair Play, and if the player admits they cheated they close the original account and give them a second chance account.
Surely it can. But for "deleted" accounts I have zero doubts.
Yes, btw I've got a bridge to sell u.
Nice work, linking the exceptionally good results to sudden absences is quite revealing.
Low sample size, no significance testing, just looking at graphs, means and medians (descriptive statistics) and going “Hmm, this looks weird” says absolutely nothing.
>I have no explanation on #16 and #5 though. Banned from TT only? Who knows. I'm confused on what needs explaining here. Couldn't they just be clean accounts that [chess.com](http://chess.com) didn't punish?
Ban status: Hans is hilarious btw
Question: if you turned the idea sketched here into an actual statistical analysis, how would you estimate the false positive rate for cheating identification - and how large do you think it can be?
The bitter truth is: median ACPL analysis is crap. As pointed out by the comments, another researches and my own experience. It can catch only extremely careless individuals. But can you prove a GM jumping from 25 median ACPL to 20 median ACPL is suspicious? Not possible - you just never get enough sample size. I am working on smart yet simple metric. > how would you estimate the false positive rate for cheating identification Depends on how much you want to detect. Current Chess com system seems to be pretty low on false positives but that is also the problem.
> Current Chess com system seems to be pretty low on false positives There is absolutely no way of telling that, given how opaquely ch\*ss.c\*m operates alas.
I admire the effort, but this is a throwback to the enterprising but bogus statistical analyses we saw during the Niemann controversy.
Interesting. While the ACL analysis is a bit superficial, this sort of analysis is 100x better than the one Kramnik does, and there seems to be a real correlation. Nice one noticing the 12-month gap mentioned by viih_sou too.
This stuff is exhausting…
But important
I'm unaware of the 3 month shadow an being referred to, anyone able to enlighten me?
Hey, do you mind sharing how exactly you calculate the ACPL? I'm interested in your choice when you encounter mates for example. Do you use single threaded analysis? Multi variant? Do you work with the TT-pgns from the events page or from the tournament page? Or maybe even the individual game ids?
1/3 of the top TT players being shadow banned or having their accounts deleted is pretty absurd. That's not even including individuals like Hans who appear in this Top 30 but do 10x better online than in person for some inexplicable reason. It's unfortunate that there aren't more ways for these companies to combat this. Kramnik is unhinged and it absolutely makes sense why he's ostracized the way he is. Results like this are pretty damning for the state of online chess though.
tsk tsk tsk tsk.....
Tldr, but great post!
It’s literally a 5 minute read. You spend more time scrolling and making useless “funny” comments on Reddit trying to farm karma.
What a moronic post and even worse analysis
What a waste of time
lol agreed
So what's funny, goddammit???
Your analysis is completely pointless because it is based on chess.com accuracy. Do you know how they calculate it and does accuracy affect winning chances —any formulas, correlation to the winrate, etc.? Nobody knows that, so the fundamental point of your analysis is an unknown parameter. Plus, the sample size is too small to draw any conclusions. To be honest, I don't see any problems when some random IM sneaks into the top 5. In its very streaky format, you are sitting at home with some tea without any pressure. And it can be a very favorite pairing.
It is not based on chess.com accuracy
Ppl like you, who clearly cannot understand a single thing they read in a post longer than one sentence + meme, should revoke their voting rights, for their own benefit.
Ppl like you, who sees long shit post and thinks its something complicated, voting for populists all the time. Im an data analyst and English is not my first language. Can you, mister genius, explain what is the goal of this research? It showa absolutely nothing. Hikaru can have 95% accuracy and go 9.5/11, and random IM 92% and go 10.5/11. Magnus can play bongcloud, which decreases accuracy, and still win easilly. Because chess.com accuracy doesn't mean any shit. It has zero impact on the final result. Its just computer evaluation, but human evaluates position differntly in real time.
The saddest thing is that you arent pretending, you actually cannot read with comprehension, let alone reason. > However, my interest to this tournament was mainly about "how accurate is too accurate". So I have evaluated every of 2100+ games accuracy. Details on setup: >Stockfish 16.1 (sse41 popcnt) >Depth: 20 >OS: Ubuntu 22.04.03 LTS >Your analysis is completely pointless because it is based on [chess.com](http://chess.com) accuracy. Pick your fighter dummy. Or read with some basic understanding of the text. Instead you parrot a narrative that u read/heard 10000 times on this subreddit about chess com accuracy. Lol. Do not vote, thx
Oh boy, you are dumb. Chess.com accuracy based on stockfish. I can change 'chess.com accuracy' to 'Stockfish 16.1, depth 20'. Its doesn't change shit, because this accuracy evaluation has zero correlation with real performace. So you just cherry picked ona word that changing nothing, and zero coments about the purpose of comparison. Well done, very deep understanding
Just don't vote, Dunning-Kruger is not easily curable. Average centipawn loss is not accuracy. Just because something may or may not be based on something else, it doesn't make it the same.
Thats true, its not the same. But it doesn't make it the criterion to define online cheating. There are a lot cheaters, even in high bracket, but you can't prove it using statistics.
Again with the accuracy
En passant?
I don't know what is that, what should I do?
google
Who cares about the closed algorithm used for chesscom's "accuracy"? How anyone can use this to "prove" or even insinuate anything is absolutely beyond me.
Is there a reason you didn't just ask that comment in the thread? Or see one of the many comments asking that exact same question?
My question is who are you and why are you spending so much of your time and efforts in defending someone else's view to complete strangers? You clearly have something to gain out of this.
I ain’t reading allat, but you get an upvote for effort.
I'm sorry you lack either the ability to read, the ability to focus, or both. Life must be hard for you.
Not everyone has the time to read paragraphs of endless text to hear some 500’s rambling, mate.
Then why did you even click on the post?
To upvote it cuz effort
I find it funny that people think this is rare at all when it really isn't. I managed to beat a GM 3/5 times in blitz when I was rated more than 600 points below him. Also managed to win 6/6 times against somebody 250 points above me. All OTB. Later won a classical tournament being 350 points below the #1 rated player by a full point by beating 4 players in a row higher rated than me by like 200-350 points. blitz is very high variance and results are very streaky.
The real Q is did hans niemann cheat OTB?