[ameliaquining]

I don't think the next-token-predictor thing should increase anyone's confidence that LLMs aren't conscious or can't escape the control of their operators. A very closely analogous argument would "prove" that humans aren't conscious or can't do [insert task here] either. (No, I'm not saying that any of this is true of today's LLMs, I'm saying this particular argument doesn't work.)

I recommend this explanation: https://www.astralcodexten.com/p/next-token-predictor-is-an-...

[dgellow]

You can say that for any argument regarding consciousness, because we don’t have an actual, all encompassing definition of what consciousness is. In general I don’t think comparison with humans makes much sense, we should be able to discuss LLMs without always falling back to “but what about humans” (sorry for the caricature)

[ameliaquining]

Shouldn't that imply that agnosticism is the proper view, rather than asserting that something is impossible on a next-token-predictor architecture?

(Note: I don't actually think the consciousness question is the most important one in the near term. Where I think this line of reasoning gets really dangerous is when people use it to assert that LLMs can't or won't engage in certain behaviors no matter much they advance; this doesn't have anything to do with consciousness.)

[inigyou]

If you believe that matrix multiplication with random sampling is conscious, then you probably believe everything is conscious, like rocks.

Most people would expect that matrix multiplication is not conscious, and autocomplete is not conscious either.

We can't prove matrix multiplication isn't conscious, but it doesn't seem likely unless everything is conscious.

[willy_k]

There are some vague notions about consciousness emerging from complexity that some people may advance as a nuance to your point, although that just makes rocks “minimally conscious”, not necessarily unconscious. If you take the view that the entirety of our consciousness’s comes from purely classical interactions (electrical and chemical), then thats not a difficult conclusion to arrive at, but the notion that consciousness can be derived from deterministic computation does not pass the sniff test imo.

As a side note, that is why I find the idea that the brain is a quantum-classical hybrid computer appealing. And following the research developments is very interesting, to say the least.

[brookst]

Seems like a forest / trees error.

“If LLMs are conscious, it means matrix multiplication is conscious” == “If humans are conscious it means cells are conscious”.

It’s possible for complex systems to have emergent properties not exhibited by any individual component of the system.

[slopinthebag]

I think you can reliably assert that X != Y without having a complete definition of Y, as long as you can identify at least one property or condition that Y possesses which X violates.

So for consciousness and LLMs it could be Qualia, lack of semantic understanding, lack of continuity in time, lack of a high degree of integrated causal feedback, etc.

Or perhaps those are just features of human consciousness but not integral to consciousness as a whole. To me this then implies panpsychism to some degree, which I'm alright with too.

[ToValueFunfetti]

I would think that qualia is the element of that list that actually matters here- "Is there a way that it feels to be an LLM?" is, I expect, the underlying question being asked within "Is an LLM conscious?". If LLMs could only experience the timeless, nonlocalized color blue, they'd still be conscious. The presence of positive and negative valence qualia is additionally relevant if someone is getting at whether they have moral worth, but it's secondary.

But qualia are not directly measurable and the rest of the list only matters if those features are necessary for qualia, which we can't decide without such measurements or at least a strong theoretical model.

[cyanydeez]

I think it's important to understand the humans _can_ do what LLMs do: predict next tokens from prior ones.

But LLMs are only operating on text and humans are only operating on <waves hands>