[prog_1]

ie when you cant beat them, make new metrics

and you can absolutely evaluate how smart someone is in a 2min casual conversation. You wont be able to tell how well they are in some niche topic, but %insert something about different flavors of intelligence and how they do not equate do subject matter expertise%

[skybrian]

It’s a common pattern that AI benchmarks get too easy, so they make new ones that are harder.