[serviceberry]

This strikes me as an odd claim. You don't hang around with a friend who makes things up because they somehow enhance your learning process. You hang around with them despite the fact they're annoyingly unreliable, presumably because you value their company for other reasons.

Let's say you're trying to get a university degree, but having a professor who makes up 20% of what they say. Is that helping you "learn well"?

[tormeh]

Once everything has made it through the jungle telephone you're lucky if it's 80% correct. 20% wrong is a downright reliable source by human standards, at least about topics which people care about.

[notTooFarGone]

But a human can tell you if they are not too sure and completely sure.

[krainboltgreene]

This is cute, but ultimately not true nor helpful.

[brandall10]

You might want to read the academic criticisms of an influential pop history book written by an academic, such as Sapiens.

And 20% is way overstated, esp for a SOTA model when it comes to verifiable facts.

[serviceberry]

I don't think that's a useful comparison. Humans writing history books have agendas and biases, but they're usually fairly transparent. In contrast, LLM failure modes are more or less impenetrable and very non-human. You're just inexplicably served with some very convincing but incorrect stuff.

[ismailmaj]

20% is a harsh figure but it could be a good entry point to figure out the unknown unknowns and go in depth once you have the relevant keywords using more reliable sources.

[edgineer]

Well that sounds like oral history, which is how all people used to learn. Strictly fact check everything you say seems like a modern invention.