[koalala]

From the article: A "stochastic parrot", according to Bender, is an entity "for haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning."

It seems to me that the great success transformers are now enjoying is precisely due to the fact that 'probabilistic information about how they combine' _is_ meaning.

[NoGravitas]

It's really not. Read the National Library of Thailand thought experiment to understand the difference. But this isn't saying that AGI is impossible, only that it can't come purely from LLMs, and that pure LLMs will remain stochastic parrots no matter how they are scaled up.

[IshKebab]

I agree. There's a quote in that paper about how ML models can never access meaning (semantics of words) because they only see the form (syntax and letters) and the two are somehow completely divorced.

It's obvious nonsense. I can describe a new concept to you using only words and letters and you can understand it. Therefore you can build up knowledge using only syntax.

Nobody is saying that LLMs understand the layout of a bus or the feel of leather, but they understand that buses are vehicles with four wheels that transport people etc.

Face-slappingly poor philosophy.