[czzr]

This is fascinating to read through. It’s so hard to avoid a variant of the Forer effect, though - where we unconsciously discount the errors, and selectively focus on subsets of the output and impute meaning to them.

Designing objective quality tests must be an active area of research, I wonder what the best approaches are?

[gwern]

Forer effect material was written by humans, one might note... But there was a paper just the other day on more rigorous evaluation: https://arxiv.org/abs/2006.14799

[czzr]

Thanks for the link. You’re right that Forer effect material was written by humans, but the point is more that there is a failure mode in our thinking that can be exploited - mostly intentionally by “psychics”, unintentionally by automated text generators.

Just something I was mulling over, though, not to take away from the obvious progress here.