[TaupeRanger]

Whether Bender intended it or not, the term has an inherently pejorative sense. "Parroting" is not really indicative of what modern LLMs do. However, when most people bring it up as a criticism of "AI in general" in 2026, they're using it as a pointer to all of the social/environmental criticisms, rather than the technological capabilities.

[tptacek]

We're all going to stay fixated on what exactly a "parrot" is, but the paper doesn't rest on the definition of tropical birds: it advances a specific and detailed hypothesis about how language models don't operate with communicative intent, and in fact the meaning humans obtain from LLM output is a cognitive pattern matching illusion. Those claims have not held up at all.

At this point, I think the authors are really counting on people not to read their paper.

[thomastjeffery]

How can you articulate a criticism without a repetition of your criticism becoming "inherently pejorative"?

You can't. That's just the way the news goes.

Essentially, what you are saying is that because some people somewhere frame a statement as pejorative, the statement itself is inherently pejorative. By that logic, every criticism ever articulated is inherently an insult.

[delis-thumbs-7e]

Why is it not indicative of what LLM’s do?

[naasking]

Because they don't just parrot, they interpolate, which is why they have such varied abilities. You can't explain the range of behaviours they have with just parroting, and once you accept that, why shouldn't this qualify as some form of intelligence?

[delis-thumbs-7e]

There’s a lot of confusion here on what parrots exactly are capable of doing and assumptions how smart they are. I also don’t quite see what you mean by behaviour in respect to LLM’s, possibly “agentic” tool usage? Because I’m fairly certain you can explain their behaviour as “stochastic parroting” combined with very strong program, ie. harness, to interact with other systems. And perhaps this all is “some kind of intelligence”, but then you have to be very careful with what you mean by intelligence. It becomes a terribly slippery surface described like this and you see, who knows where this gradient descent takes us if we are not careful.

So personally I would still rather undersell it. But parrot or not, it is still a terribly useful little bird indeed.

[TaupeRanger]

Really? Are you under the impression that parrots are able to synthesize their input and create entirely new, useful outputs which they have never heard before?

[softwaredoug]

The "stochastic" is the key modifier to "parrot".

> "Stochastic" means having a random pattern or variable that can be analyzed statistically but not predicted precisely

[delis-thumbs-7e]

Yes, I am: https://en.wikipedia.org/wiki/Alex_(parrot)

[inspectorSlap]

I think stochastic parroting is really a very accurate description of what they do (if underserving of the overall usefulness of LLMs). As long as you consider they are parroting from the whole of human intelligence. Its just that as they have gotten more sophisticated, the amount of gates, guardrails, and tertiary tools add variety. Trace any LLM hallucination back to provenance and you begin to see how the stochastic parrot works.

[koverstreet]

Yeah, there's some beautiful math underlying what LLMs are doing, and it's the same math our neocortex runs on.

[dekhn]

There's plenty of beautiful math there, but the relationship to what our neocortex does is pretty distant. Individual biological neurons can do fairly complicated things, including compute 10-bit parity functions (you would normally need a 3-layer MLP with a bunch of digital neurons to do this). And they don't seem to use backpropagation for learning.

[ainch]

Could you provide more detail? My understanding is that the neocortex is predominantly focused on forwards simulation, which seems distinct to how transformers operate.

[koverstreet]

That's fundamental to how anything that compresses/understands the world has to work, in the Kolmogeravian sense. That's why people denigrate LLMs as being just "next token predictors" - they're not wrong, but they're missing the point.

Because to do that kind of prediction out in the world you have to build up an accurate model of reality - a model that includes yourself! Which is why we and LLMs are self aware.

For the "how", it's been known for some time that LLMs operate on a Reimannian manifold - the semantic manifold - and that's a good place to start if you want to learn how they actually work; how a Reimannian manifold (plus some extra structure on top) can represent natural language in a form you can do work with is the part I find particularly beautiful. At a high level, the neocortex and LLMs appear to compute on the manifold in basically the same way - though a lot of the details are different; both are more sophisticated in some areas and less in others.

[ainch]

I'm afraid the precise connection you're making isn't totally obvious to me.

As far as prediction - I mean sure the cortex and LLMs do prediction, but then so can RNNs or diffusion models or any other generative model. Really any ML architecture is learning to compress its environment in pursuit of modelling. More broadly, the predictive brain model would suggest that all of the brain, not just the neocortex, is dedicated to prediction. What would you say makes LLMs similar to the neocortex, rather than the basal ganglia or Broca's area?

Similarly, if you agree with the Manifold Hypothesis, then all machine learning models operate on manifolds. I agree it's an exciting thought, but then I don't know what would distinguish an LLM from a VAE or SVM in terms of operating over a low-dimensional manifold embedded in high dimensional spaces - maybe just scale?

[koverstreet]

> More broadly, the predictive brain model would suggest that all of the brain, not just the neocortex, is dedicated to prediction. What would you say makes LLMs similar to the neocortex, rather than the basal ganglia or Broca's area?

The whole brain is most definitely not dedicated to prediction, and I don't think "prediction" is a very useful model anyways. You could say the hippocampus is for "prediction" if you really squint, but that's underselling what it's doing. And the basal ganglia operates off of prediction error, but it's more about regulating short timescale feedback loop learning than learning itself - speaking somewhat loosely because it's a particularly ancient structure in the brain and things get muddied.

LLMs obviously don't have functional equivalents to either of those - sure they can remember facts, but they can't intake new facts the way we can, the hippocampus is set up completely differently. But higher reasoning is humans is the neocortex, and that LLMs can do, and there's even structural parallels - LLMs and the neocortex are both implemented in layers, and you can even (with a shit ton of analysis) watch how LLMs walk the geometry across layers. The neocortex needs a lot fewer layers than LLMs, but that's because it's not pure feedforward across layers; the neocortex already does what the "looped LLM" people are trying to do.

> Similarly, if you agree with the Manifold Hypothesis, then all machine learning models operate on manifolds. I agree it's an exciting thought, but then I don't know what would distinguish an LLM from a VAE or SVM in terms of operating over a low-dimensional manifold embedded in high dimensional spaces - maybe just scale?

I view things more mathematically, and in math and physics it frequently turns out that there's really only one solution to a problem, or correct way to model something - and then everything else is just isomorphisms (which may be interesting objects in their own right!).

My interpretation/guess (informed by research, but with a lot of still unanswered questions) is that the manifold is fundamentally the structure that emerges from natural language: natural language has a computational model, just like Church/Turing machines have a computational model. Reimannian manifold is also underselling it a bit, it's also Finsler, and RoPE adds additional structure to that manifold (and when we get to RoPE it wouldn't surprise me if the neocortex and LLMs do diverge significantly in how they compute, but does the structure change? dunno yet).

Basically, I'm fairly confident in what I say about machine learning models that understand natural language; for non-NL machine learning, maybe the same structure emerges with enough complexity, but I haven't really pondered that.

[ainch]

Thank you for expanding! I come from more of an ML background so still learning a lot on these topics:

Agreed that the neocortex uses fewer layers because of looping - I also suspect it's partly because neurons are more complex than the neurons used in ANNs, so in principle they should be capable of more sophisticated computation in a single forward pass (especially considering that handling multiple neurotransmitters could mean superposed functions).

The point about there being one correct way to model something does seem backed up by the Platonic Representation hypothesis (https://arxiv.org/pdf/2405.07987). I've even seem some work that shows you can find a bijective map between the latent spaces of different transformers (https://arxiv.org/pdf/2505.12540).

Your point about Finsler spaces is fascinating - I hadn't come across the term before. It'd be interesting to see if there's work that specifically allows latents to exhibit that kind of directional metric behaviour, and whether that improves generalisation or something.

On the entire brain being dedicated to prediction, that's more of a high-level comment. I was thinking of work like Andy Clark on predictive processing, which suggests that even regions of the brain which we think of as receptive (eg. the visual cortex) may be implementing a generative/predictive model which is corrected by sensory input.

[burnte]

> and it's the same math our neocortex runs on.

The math is based upon theories of how the brain work, but even if those theories are right, this math is a great simplification and subset of what organic brains do.