[flir]

> You can have the model call you out for referring to the model as existing.

This tickled me. "There ain't nobody here but us chickens".

I have other thoughts which are not quite crystalized, but I think UX might be having an outsized effect here.

[galangalalgol]

In addition to he/she etc. there is a need for a button for no pronouns. "Stop confusing metacognition for conscious experience or qualia!" doesn't fit well. The UX for these models is extremely malleable. The responses are misleading mostly to the extent the prompts were already misled. The sorts of responses that arise from ignorant prompts are those found within the training data in the context of ignorant questions. This tends to make them ignorant as well. There are absolutely stupid questions.