[api]

I’d argue that all these models are stochastic parrots because they’re not embodied in any way. There is no way they can actually understand what they are talking about in any way that is tied back to the physical world.

What these LLMs and diffusion models and such actually are is a lossy compression method that permits structural queries. The fact that they can learn structure as well as content allows them to reason as well, but only to the extent that the rules they’re following existed somewhere in the training data and its structure.

If one were given access to senses and memory and feedback mechanisms and learned language that way, it might be considered actually intelligent or even sentient if it exhibited autonomy and value judgments.

[danbruc]

I’d argue that all these models are stochastic parrots because they’re not embodied in any way.

I do not think that this would really change much in itself. If you tell the model that crimson is a shade of green, it will learn something wrong whether it has a body or not. What you need is feedback on whether a response is correct or not, factually correct, not grammatically correct. Alternatively you have to teach the model to perform its own fact checking and apply it to its responses.

[AnimalMuppet]

I think that maybe "truly understand" is anchored in the physical world. I don't exhaustively know what, say, grass is, but I know what it looks like, and I know what it feels like to walk on, and I know what it feels like to touch with my hands, and I know what it sounds like when I walk on it, and I know what it smells like when it's cut. And I know that there's a consistent correlation between "stuff that look like that" and "stuff that smells like that when it's cut".

And so if the topic of grass comes up, I have some firsthand knowledge to draw on - less than a botanist, but not nothing. I have some sense impressions that correlate to other sense impressions and to the word "grass". GPT, on the other hand, has some words that correlate to other words, and nothing more.

So it seems fair to say that I understand grass on a level that GPT does not, and cannot. Therefore it seems fair to say that GPT is at least closer to being a stochastic parrot than humans are.

[grumbel]

And yet if you see some AstroTurf you'd still call it grass. In the end there is no "true understanding", there are just predictions we make about the world. Depending on how deeply you look, they are often incorrect, but also generally good enough.

GPT isn't quite at the good-enough point and being limited to only text, makes it impossible to reason about aspects of the world that are difficult to describe in text or simply weren't in the training data.

And more generally speaking, the claim that LLMs don't understand anything really doesn't hold up given how much they are able to hallucinate. If a LLM truly wouldn't understand anything, it wouldn't be able to generate plausible text, it would either generate nonsense or be limited to whatever was in the training data, but that's not the case. The LLMs can predict past their trained knowledge and predict stuff they haven't seen yet. Those predictions will sometimes turn out wrong, but so will the humans prediction that the AstroTurf is grass when taking a closer look.

[danbruc]

Sure, you have experienced grass with more senses than just reading about it, but I do not think that this fundamentally changes anything. If I lied to you your entire life and told you that you are walking on or smelling grass while you were actually walking on moss, you would learn a similar mistake spanning several of your senses.

[AnimalMuppet]

No, I would know moss instead of grass, and I would know it consistently. I would simply have a different label stuck on it than everyone else used.

[chrisnight]

The idea that an entity can't be sentient because of a lack of senses has the problem where it invalidates the sentience of humans though. Do you consider a blind person having less sentience than a person that can see because they lack the sense of sight? Even if we consider sentience as an on/off switch, what about a person that has no senses at all (whether someone like that exists theoretically or in reality)? With no way to tie back thoughts to the real world, are they no longer sentient?

Obviously we don't know for certain if other humans are sentient, but it seems necessary to establish the premise they are in order to get anywhere in the argument for sentience of AIs. In this case, we need an argument about the sentience of AIs that coincides with our experiences of the sentience of humans, which this argument doesn't seem to do.

Even if we limit ourselves to thinking about people with all of their senses, there's still information that we cannot tie back to the physical world with our senses. Take someone who sits at a computer all day. They read news and talk about it online, without ever interacting with the news physically. Take someone who theoretically has never done anything outside of read and type on a computer all day. Are they not sentient because they've never physically interacted with the world outside of their computer?

[grumbel]

> Do you consider a blind person having less sentience than a person that can see because they lack the sense of sight?

They still interact with an external world. An LLM doesn't, at all, not even a little bit. That's the crucial difference. A person will know when things didn't go as predicted, as the real world will provide feedback they can sense. An LLM in contrast has no idea what is going on, its past actions don't exist for it. There is only the prompt and the unchanging base model.

That said, this is not to disparage the abilities of LLMs, they simply were never designed to be sentient. If one wants an LLM that is sentient, one has to build some feedback into the system that allows it to change and evolve depending on its past actions.

[denial]

A syntax-producing machine harnessing the power of duality will still never have access to semantic content. For this reason I have difficulty saying that it understands things beyond a colloquial sense.

[jstx1]

> if it exhibited autonomy and value judgments.

Who wants this from ML systems? I want them to be useful, not to have autonomy and value judgments.

[grumbel]

It has to have some degree of autonomy to be useful. The current approach with ChatGPT to just have all the knowledge in the world directly in the base model not only doesn't scale, it would also run into issues with copyright if it could actually recite books and stuff word for word. A ChatGPT that can just use Google to look up the necessary information itself would be far more useful.

BingChat sort of tries that, but it doesn't really have any autonomy either, so it just summarizes the first Bing search result it gets. It would be far more useful if it could search around two or three layers depth into the search results to actually find what you are looking for.

In general current AI systems have the problem that you have to babysit them far to much. If you want to get specific answers, it's you that has to provide all the necessary context to make it happen, the AI can't figure out by itself what you want from past conversations.

[kelseyfrog]

I have a few projects in mind where that's a requirement.