[nonfamous]

This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human.

What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?

[pegasus]

What? Where are you seeing such claims? I.e. that only humans can be conscious, that AI in general is not conscious by definition?? Seems to me you've put up an easy-to-dismiss strawman.

Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective.

IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.

[nonfamous]

Err… right here in TFA?

>> The first requirement is that the computer program has a body (either physical or virtual) and sense organs

LLMs may be chameleons, but, to steal an excellent quote, if you can’t tell does it really matter.

My personal take on this is that consciousness is a thing induced, not a thing evoked. If it generates a response on the observer as a conscious thing would it is, by my definition, conscious.

[pegasus]

Sure, by your definition they totally are conscious simply because at one point in the past they fooled one person that they might be so (and so is the OG chat agent, ELIZA). But no one else will accept such a definition.

And if you read my comment above carefully, I point out at least one way you could tell whether an agent has an I or just pretending to – by tracking their energy expenditure/reaction times. But they would give themselves in other ways as well. Pretending can only fool one short-term.

I'm not going to discuss the quote since @sp1nningaway already pointed out it doesn't say what you say it does.

[sp1nningaway]

If TFA says "only humans can be conscious," you may know many more humans that have a virtual body made up of virtual organs than I do.

[voidhorse]

The argument doesn't state that at all. It very clearly admits consciousness to nonhuman animals, for example.

[pnexk]

Another example of how we’re all collectively becoming worse at properly reading and parsing longer text.

Really does make me wonder about backtracking on our breakthroughs in literacy rates.

[partyficial]

why does AI/LLM deserve special scrutiny regarding consciousness or the lack of?

why not publish a paper stating - 'the radio is not conscious just because it can speak words'. or "the phone is not conscious because it can speak and listen".