[WorldMaker]

The quantity has certainly changed. It took researchers months to build an "AI generated novel" way back in the day and some form or relative of ChatGPT spits out nearly that every minute now.

I still haven't felt impressed that the quality has truly changed, yet. LLMs seems more "fluent" in the language than ever before, but it's still hallucinating nearly as much and now the fluency just helps make people more often see "meaning" or "anthropic action" (lies, defamation) where the hallucinations are. The underlying structures of LLMs are still complicated casinos that invoke the Gambler's Fallacy much more than any signs of true "learning". We've put millions of monkeys in front of billions and trillions of slot machines and told them to produce Shakespeare and many of them believe they are doing just that. (Not just metaphorically, by monkeys I mean as much humans susceptible to casino payout mechanics and excitedly spinning slot machines.)

Again, yes, I'm a terrible cynic right now, and I hate to be so down on the technology, but I'm still waiting for something to be excited about that isn't just casinos masquerading as "learning". But people love casinos, they deliver addictive fun. I'm not going to stop people from being excited about all these casinos. I just think that professionally as a software developer, if I wanted to be a bad faith casino manager I'd rather just get into mobile games and gacha/loot-box mechanics. That's more fun, more profitable, and maybe, weirdly, more currently ethical than current "generative AI" hype.

[int_19h]

The emphasis on "hallucinations" is misplaced from this perspective, IMO. Thing is, when models do hallucinate, they still reason about what they hallucinated. Larger ones (e.g. GPT-4) can even spot their own hallucinations. That is nothing like what we had in the 60s, or even 10 years ago.

[WorldMaker]

I dislike the term "hallucinations" because I feel it also anthropomorphizes the process too much. Unfortunately, "random garbage output" is too many words, but that's closer to what I meant everywhere I used that word.

> Larger ones (e.g. GPT-4) can even spot their own hallucinations.

I've not yet been convinced that this is actually what is happening from the examples I've seen. It all looks to me like more "random garbage output" that "feels correct" but isn't provably correct. Most examples I've seen so far look too much like "Stochastic Crow Mode" [1]. It is prompts and questions that are doing much more work on the humans reading them (and our interests in anthropomorphizing them or mythologizing them) than the LLMs answering them.

[1] https://fediscience.org/@ct_bergstrom/110182336553459017