[Legend2440]

Yeah, I think it's pretty clear that LLMs are more than mere "stochastic parrots" - they can prove theorems, follow instructions, and complete complex tasks.

This was the most notable claim of the paper, and it's aged very poorly.

[plastic-enjoyer]

Are they, though? I think what LLMs proved is that proving theorems, following instructions and solving complex problems - intelligent behaviour - does not need any kind of understanding, but only ability to recombine things in a stochastic matter. Which basically just means that these things weren't as special as people had thought.

[tptacek]

We've clearly crossed a threshold at which "stochastic" is no longer doing the work Gebru (and, more importantly, the acolytes of this paper; I shouldn't tar Gebru with what they've done with the work) expected it to do. Lots of important processes are stochastic, including at some levels human thought itself. Advocates who deploy the term "stochastic" seem to believe it impeaches the technology, which is kind of embarrassing to see.

[plastic-enjoyer]

> We've clearly crossed a threshold at which "stochastic" is no longer doing the work

What do you mean?

[tptacek]

An example of the loading the term "stochastic" has to Gebru: the paper goes on at some length about how the coherence of ChatGPT responses is in part a product of human pattern-matching instinct, that we're primed to see coherent responses whether or not there's truly a communicative intent behind what we're reading. That insinuation hasn't held up at all! It is not a failure mode of modern frontier models (or the last several generations of models) that they routinely collapse into gibberish revealing the messages they've sent to be meaningless the whole time.

Nonetheless, despite the fact that GPT 4o could reliably solve randomly generated multivariable calculus problems, these systems are at bottom still fundamentally stochastic at least in their kernels (you could have a philosophical debate about how stochastic the entire training process is given how dependent it is on RL). So what does it tell us that an LLM is "stochastic"? About as much as we could glean from the knowledge that the signaling in the computer systems we happen to be using right now is "electronic". It's an interesting fact about the world, but not something especially helpful to make predictions from.

I think Gebru --- or at least, the abstraction of Gebru I formed in my head after reading this one paper --- is probably surprised by that outcome. Surprise is good and healthy! The acolytes, though, who Gebru is not responsible for, are something worse than surprised.

[plastic-enjoyer]

> So what does it tell us that an LLM is "stochastic"? About as much as we could glean from the knowledge that the signaling in the computer systems we happen to be using right now is "electronic". It's an interesting fact about the world, but not something especially helpful to make predictions from.

I think we've been talking past each other. The term “parrot” may do a disservice to AI, I think, however, that one can go so far as to say that AI is a stochastic recombinator that has the potential to solve complex problems. And I do think that this a pretty interesting thing that goes above being just an interesting fact about the world, since it reveals quite a bit about what we have considered to be special to us is not so special, namely, that reasoning and complex problem-solving may not require understanding at all, but can be achieved through pure stochastics. This may not help you with making predictions, but I think that anyone with a curious mind should also be interested in the implications for our view of humanity.

[tptacek]

I don't think we're talking past each other, I just think we're struggling to find a disagreement. All I'm saying is that anti-AI advocates (and the Gebru paper, by implication) refer to the stochasticity of LLMs as a core limitation, and that's a category error.

[Legend2440]

I think you have already decided that LLMs cannot possibly understand. Therefore anything they do must not have required understanding in the first place. It's circular logic.

[plastic-enjoyer]

> I think you have already decided that LLMs cannot possibly understand.

Well, maybe you should stop thinking.