[tivert]

> LLMs are best thought of as language generators (or "writers") not as repositories of knowledge and facts.

And the utility of a "language generator" without reliable knowledge or facts is extremely limited. The technical term for that kind of language is bullshit.

> People don't care about (or are willing to accept) the "wrong answers" because there are enough use cases for "writing" that don't require factual accuracy. (see for instance, the entire genre of fiction writing)

Fiction, or at least good fiction, requires factual accuracy, just not the kind of factual accuracy you recalling stuff from an encyclopedia. For instance: factual accuracy about what it was like to live in the world in a certain time or place, so you can create a believable setting; or about human psychology, so you can create believable characters.

[yaj54]

I'd argue that what you're talking about in fiction is coherence (internal consistency) not factual accuracy (consistency with an externally verifiably ground truth).

I'd also argue that the economic value of coherent bullshit is ... quite high. Many people have made careers out of producing coherent bullshit (some even with incoherent bullshit :-).

Of course, in the long run, factual accuracy has more economic value than bullshit.

[tivert]

> I'd argue that what you're talking about in fiction is coherence (internal consistency) not factual accuracy (consistency with an externally verifiably ground truth).

No. I'm talking about "factual accuracy (consistency with an externally verifiably ground truth)." Mere internal consistency is not enough: a fictional world where everyone consistently stabs themselves in the eye when they see flashing lights is consistent, but lacks factual accuracy, and is therefore garbage fiction.

> I'd also argue that the economic value of coherent bullshit is ... quite high. Many people have made careers out of producing coherent bullshit (some even with incoherent bullshit :-).

I agree there's (greedily selfish) "economic value" to coherent bullshit, but there's negative social value to it. It's basically a kind of scam.

IMHO, some of the best applications for LLMs are for things like spam and scams, not the utopian BS they're promoted for (e.g. some LLM will diagnose your illness better, faster, and cheaper than a doctor).