Framing & launching LLMs as a "chat" interface is the source of many ills. I don't have a simple solution, but leaning away from conversational interfaces would lead to less anthropomorphizing.
I feel "document generator" is the best and most-grounded framing.
Right now, lots of people get caught in a trap of asking things like "does MyFreeBestFriendAI feel remorse?"
If we're already looking at it as a document artifact, we can evade the implied-ego trap: The document generator took a chat-like document where two fictional characters are talking, and predicted that there would be text where one fictional character is associated with apologetic words.
One aspect is that chat LLMs are trained to talk like a person - "If I should do X, just say the word" etc.
It would be interesting to train an LLM that consistently "talks like a computer" or a command line utility instead, i.e. passive sentences, relatively bare results of the tasks given, no reference to a self, etc.
There's a webcomic where the author wrote some backstory before LLMs, where the world over-trusted in statistical models until it went wrong in a ghastly "stupid Skynet" way.
This led to international requirements that all "AI" have explainable/auditable decisions, and must have clear non-human features.
Right now, lots of people get caught in a trap of asking things like "does MyFreeBestFriendAI feel remorse?"
If we're already looking at it as a document artifact, we can evade the implied-ego trap: The document generator took a chat-like document where two fictional characters are talking, and predicted that there would be text where one fictional character is associated with apologetic words.