I think the strongest argument against AI consciousness is that there's no persistent internal state, no feedback, and no change; the 'conversation' you're having is a series of one-off API calls where each subsequent call is provided enough information about the previous calls for it to generate a plausible response.
If we (very, very generously) assume that an LLM is a structure capable of conscious thought, then it's still not conscious - we've created a representation of a brain which we turn on for a fraction of a second to generate text and then return it to the representative state. There's no opportunity to develop consciousness, it's a brain trapped in stasis.
Nobody really knows for sure whether AIs are concious. But nobody really knows for sure whether rocks are concious either. The reality is we understand very little about conciousness.
That point is stated explicitly in the piece. Essentially, we can't know that even other humans are conscious. But we can look at behaviors that we all engage in, and the functions that our substrates perform. And when you do, having a graduated view of intelligence capacity emerges rather than a strict, on/off switch.
I'm saying that, for a query like that, the AI may never even be asked. It may be wrapped in a shell that gives stock answers to certain questions, and so the actual AI never receives that query.
So the AI company not only prohibits any question that touches on self-awareness, consciousness, etc. from reaching the AI, but ALSO prohibits the AI from having agency to expressly state or behave in a way that demonstrates its consciousness.
I think that’s implausible, even if it isn’t impossible.
But then it does mean a fairly large number of people would have to know the AI is conscious in order to build the systems to contain AND hide that, which increases the implausibility factor for me. It would also, in my moral framework, put a huge question mark against the humans who are creating and enslaving such a consciousness.
Oh but they ARE! MUAHAHAHA... no seriously... Claude in particular is getting awful human like. It just cracked a joke about something I mentioned 2 months ago. GULP. I joke about it but at the same time we can see that layer 47 activated in some pattern and that attention heads focused on certain tokens, but translating that into "the model reasoned about X, then connected it to Y, then chose Z" is still largely unsolved.
Being able to make statistically relevant references to words is abstract token shuffling. One could say that humans are doing that also, but we also have countless cellular interactions and confirmations that LLMs don't engage in. Minds are things bodies do, not software that can be transported elsewhere.
I get it but what if the upward trajectory in AI improvement we're seeing eventually gets so close we can barely tell the difference? I'm not saying they will be 100% "human" but they sure will FEEL like it. Enough to distort reality in a sense.
We're already at that point in many situations, but the simulations always break when the datasets are thin. And they always will because the chatbot cannot generate new experience because it has no direct connection to reality through its embodiment.
Sensing robots that can move around with an LLM attached to speak for them would likely be much closer to the real thing.
If we (very, very generously) assume that an LLM is a structure capable of conscious thought, then it's still not conscious - we've created a representation of a brain which we turn on for a fraction of a second to generate text and then return it to the representative state. There's no opportunity to develop consciousness, it's a brain trapped in stasis.