[Groxx]

I generally like it still. It describes their failure modes pretty well, and in a way that most people already recognize. They're incomparably more complex, of course, but they are not intelligent and they are very much repeating what they've seen without any capability for factual accuracy.

Practically every other attempt at describing them leans too technical and unfamiliar (stochastic parrot) or too anthropomorphic (even describing them as "not like a human" gets people thinking in terms of humans, like how if I mention that your tongue is in your mouth all the time, using up almost all of the room, feeling your teeth and tasting itself, you're now uncomfortably aware of it and the numerous bumps on the surface).

You need to work from a reference that has both a shared understanding, and does not lead to problematic "if X has Y, and Z is like X, then Z has Y" seemingly-logical derived beliefs. "Spicy autocomplete" is a fairly safe starting point in both ways.

[red75prime]

"Repeating what they've seen" could be, depending on their generalization abilities, something like low-n-gram Markov chain, repeating of surface-level speech patterns, repeating reasoning patterns, repeating mind-theory-level patterns, repeating self-correction patterns and so on.

[Groxx]

And they fall somewhere between "surface-level speech patterns" and "repeating reasoning patterns" imo, yeah. They do indeed learn higher-order patterns than a Markov chain... but they'll blatantly misapply reasoning patterns and self-contradict confidently, just like you see with human bullshitters who don't understand one bit of what they're doing.

They sound fairly human, until you notice the patterns. They sound like they're thinking, until you pay attention.