[mcguire]

Who are we talking to when we talk to these bots?

https://medium.com/@colin.fraser/who-are-we-talking-to-when-...

"It is an intentional choice to present this technology via this anthropomorphic chat interface, this person costume. If there were an artificial general intelligence in a cage, it feels natural that we might interact with it through something like a chat interface. It’s easy to flip that around, to think that there’s a technology that you interact with through a chat interface that it must be an artificial general intelligence. But as you can clearly see from interacting with the LLM directly via the Playground interface, the chat is just a layer of smoke and mirrors to strengthen the illusion of a conversational partner."

[ekidd]

First and foremost, a GPT is an improv actor. Given a text, it's trained to continue the text as naturally as possible. This is not terribly useful for many tasks. And like an improv actor, if it doesn't know what should come next, it will make up something that sounds good.

Next, our universal improv actor is trained to play a specific role: someone who answers questions. But not just any questions, because it freaks people out if they ask the AI for advice and it replies "You could accomplish your goals by assassinating these 6 real people, and here's why." So the universal improv actor is trained to play a question answerer who gives harmless advice.

But to get any work out of the models, they need to know what role to play. And "someone who tries to respond to questions" is a flexible role, and one which allows responses to be further customized.

In other words, the conversational interface is 50% because it's a self-explanatory UI, and 50% for the benefit of the model itself, to nudge it into playing a useful role.