[salamo]

ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one.

This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely.

No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.)

This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though.

[1] https://arxiv.org/abs/2104.07567