[wolverine876]

> High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters.

That's similar to people describing how to catch other frauds, such as fake Amazon comments or bots. It's medieval 'science': They usually have no evidence of their accuracy, either false negatives (frauds who they overlook) and false positives (people falsely accused of fraud). So it's easy to say, 'this is how to identify them' - nobody will ever test your claim.

Regarding false negatives, for example, there is reason to believe that people detect only the obvious frauds, and that our detection becomes tuned for the obvious. Regarding false positives, people will cite the 'obvious' positives - e.g., some humanly impossible property - but even if they are correct, the problem is the cases in the grey area. False accusations are no joke.

Ironically, now we want a bot to solve our problems. What data do we have to say that it's accurate, or any more accurate than we are?

[mintplant]

> It's medieval 'science': They usually have no evidence of their accuracy,

I mean, not really. The difference between human and computer chess play-styles is well-documented, to the extent that in the earlier days of chess engines, human chess disciplines were developed to counter the way computers play ("anti-computer chess").

[robbintt]

If it makes you feel better (or worse), signal "fingerprinting" is used in laboratory science to verify things like purity and identity. Detecting a lack of divergence from a known chess bot seems like a good fingerprint to me.

[than3]

People have been asking for a transparency report to verify accuracy for various countermeasures for awhile. The platforms simply won't release that kind of information (chess.com & lichess.com)

They have an incentive to show their game isn't a den of cheaters, and yet they don't release which means there is a stronger incentive to hide that information.

Makes you wonder what kind of incentives are preventing them from releasing that information. Marketing says 100's of millions of games. Are they games between two people, or potentially a lot of matches against computers (where you don't know they are computers up front). Food for thought.

[ConstantVigil]

Thing is... if your methods work well enough to help you avoid the thing you wish to avoid; does it matter if it's right or wrong? It works, right?

[wolverine876]

> if your methods work well enough to help you avoid the thing you wish to avoid; does it matter if it's right or wrong?

You don't know if it's helping you at all; that's the issue. The latter question is a bit bizarre.