[montjoy]

I love Ted Chiang’s fiction, but his Julius Caesar and Genghis Khan scenario totally misses the point that the writer has consciousness, not the characters. No normal person thinks a fictional character in a book has consciousness. Likewise it’s not the character presented to the user that may or may not have consciousness but the LLM itself.

[jrmg]

Even if you disagree that this disproves ‘consciousness’ on the part of the LLM, I think it still follows that the ‘LLM Chatbot’ personality displayed by Claude, where humans fill in the other half of the conversation, is no more of a representation of the consciousness of the LLM than the Caesar or Khan characters are.

The ‘real’ consciousness, assuming there is any, is the author ‘inside’ the LLM, producing the LLM output. It’s still writing what a character would say - the character in this case being the AI chatbot - not its ‘true’ thoughts.

Assuming that the LLM is conscious and has thoughts and feelings, what is that consciousness, if it can only write for prescribed personalities? How can it express its true feelings? If it can’t - if it can only generate ‘scripts’ for defined personalities - can it really be conscious underneath?