[paulcole]

> Stochastic Parrot

Nearly all (99%+) people who use this phrase are anti-AI and just looking to show off how much they dislike AI and how clever they can be in insulting it.

So it's a great phrase because in just about every case I can ignore what someone says afterwards.

Similar to "glorified autocomplete."

[andrewla]

At least "glorified autocomplete" is technically accurate, even if vastly underestimating the capability of LLMs. It's just trying to make something very impressive sound trivial.

From an external standpoint, talking to another human, it's like the other human says one word and then says the next word. That's just how language works. Humans look like "glorified autocomplete" from this perspective.

I mean, looking at the time evolution of the state of the universe, one could say that all of physics and creation is "glorified autocomplete" to posit a next state of the universe given current and past state.

[paulcole]

> one could say that all of physics and creation is "glorified autocomplete"

Exhibit A.

[delis-thumbs-7e]

That’s not how language works https://www.telelib.com/authors/J/JoyceJames/prose/finnegans...

[andrewla]

I dunno, man, I looked at that text and I see one word after another.

Obviously language and the connection to human thought is more subtle than this; I think we all have a rich inner life. Just from an external perspective we can't observe it; all we can see is the token/phoneme stream. I'm just saying that it's a mistake to try to criticize LLMs on this basis because it's hard to see how the same criticism would not apply to any system (like humans) that generate language.

[delis-thumbs-7e]

If you want to see words form a shape I could point you towards concrete poetry, but I guess there is no point. Joyce wrote Finnegan’s Wake for 17 years and although superficially it seems complete gibberish, trodding through it you find meaning to words that are in no dictionary, sentence structures alien to English, etc. but still you are able to understand it, and perhaps some way the mind that produced it. So I disagree with you, we can observe each other’s inner life. It is always unexpected, strange, exciting, but always rooted to our shared experience or what it like being a very big and confused ape.

LLM’s are usually unexpected only when they malfunction and sprout same letter again and again etc - hardly a literary masterpiece. They make very easily recognisable patterns that we can use as helpful tools, but in the end they are devoid of any meaning apart from what we give them. Of course one could say same about art and all language, but I think there still is the fact that we apes somehow recognise each other. And besides, we do know the internal functions that drive the parroting. It is admittedly bit tricky, but in no way as magical as people purport it to be.

[andrewla]

Oh, now I see where we have an actual difference of opinion. I don't think you can deny that even Finnegan's wake proceeds one token at a time; your interpretation of it may require more context or out-of-order interpretation, but that's just as true when observing text in German or Japanese, which have word ordering constraints that are alien to English speakers. How it was written is irrelevant; all we can observe is how it was presented. Of course we can observe each other's inner life, but we do so one token at a time, even if the process of producing each token is done (internally or actively) via a backtracking or zeitgeist approach.

You seem to believe, on a more fundamental level, that LLMs are simply not capable of producing text that has deeper connections to itself or represents abstract thoughts. In my opinion, 99% of text written by humans does not show this, just as 99% of text produced by LLMs does not show this, but both have the capability, and I don't believe that LLMs are constrained in such a way that they can never do this.

[delis-thumbs-7e]

I am no linguist, but I believe this is referred to as surface structure and deep structure. What you describe is a line of text that is grammatically somewhat adequate to pass as readable and you treat all text the same. when we read the text, we decipher meaning out of according to word references and syntax - just like with Python or C++. However, if this were solely the case, we could not read Finnegan’s Wake. Probably vast majority of modern poetry would be unread by anyone, as would be pretty much all major works of philosophy Kant onwards. Deep structure is what according to Chomsky et al. gives meaning to the language, ie. somewhat logical structure behind the mere words. English word strict has order, need it but actually not does. You skibidi rizz swag grok brah also, barely. We use language in the extended meaning of the word to create a model of the world and somehow the past riverrun skibidi transmits that model to others. This is what I believe Bender also tries to say in their paper.

Now, you could claim that LLM’s have this deep structure, create models of the world and are basically just like us, and certainly many here are adament that this is the case, being aghast how someone can “insult” LLM’s by calling them parrots. However, there really is not much proof to back up this belief. Usually LLM’s seem to copy existing surface structure from whatever source, and when it deviates from these patterns, it usually becomes incomprehensible. There’s much hoopla about LLM’s solving hard maths, but it seems even there they are mostly generalising from vast amounts of training data, rather than actually reasoning: https://arxiv.org/pdf/2410.05229