[yangl1996]

They could have said "35 out of 36" instead of putting some scary number like "94%".

[axiom92]

35 out of 36 will be scarier to most people. In behavioral psychology they call this "denominator neglect."

https://www.askattest.com/blog/guides/denominator-neglect

[jszymborski]

Right, but surely sample size matters.

[cratermoon]

I shortened the title to fit HN limits. Why do you think "34 out of 36 were wrong" is less scary than "94% were wrong"?

[rhacker]

Another way to think about that: %99.99 of AIs lose chess against our best human player.

When in reality 1 AI consistently beats all Human players every time.

[cratermoon]

Only if you can go back and demonstrate that the 2 AIs that didn't fail are consistent and demonstrably correct. Otherwise they just got lucky.

[yawnxyz]

I think if they flipped it and said "Only 1 out of 36 AI systems evaluated was more accurate than ..." would have a higher impact

[qweqwweqwe-90i]

You mean 34/36

[MattGaiser]

Is that less scary?

[watwut]

35 out of 36 is scary.