[the_snooze]

I like to ask AI systems sports trivia. It's something low-stakes, easy-to-check, and for which there's a ton of good clean data out there.

It sucks at sports trivia. It will confidently return information that is straight up wrong [1]. This should be a walk in the park for an LLM, but it fails spectacularly at it. How is this useful for learning at all?

[1] https://news.ycombinator.com/item?id=43669364

[giantrobot]

But just because it's wrong about sports trivia doesn't mean it's wrong about anything else! /s [0]

[0] https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect