[tskj]

Hmm yes, I had it backwards. I agree this is very statistically significant, but the effect size is tiny. Up from random chance to a mere 60%, means that Scott proved with high statistical significant that people reliably cannot tell.

Also I'm not that worried about the adversarial conditions, any real life conditions are likely adversarial in the relevant sense. No one is one-shotting generation and serving you that, obviously output has been selected for quality. I would call Scott's test not adversarial, but fair.