[bambax]

Yes and no. It's true that when it reasons about the world ChatGPT doesn't know the truth and doesn't care, and just aims to be plausible, and it's simply bullshitting. And yes, this is the main problem with ChatGPT.

But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying.

It's also a weird personality trait, something that is unlikely to have emerged on its own but was probably programmed into it. I don't know that for sure of course and have no information about how ChatGPT was put together, but from the outside it's troubling.

[salamo]

ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one.

This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely.

No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.)

This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though.

[1] https://arxiv.org/abs/2104.07567

[afiori]

It is indeed possible that enough of its training material contains enough absurdities so that conversations like:

> is X true?

> of course, X is true.

> I did not expect you to say that X is true.

> I never said that X is true, it must be false.

to have it methodically lie.

Personally I believe that a rolling bullshit generator is more plausible; ChatGPT and GTP3 before it worked by producing plausible authoritative* answers; in its training it found out that saying true things is an efficient way not to mess up.

But otherwise it has no concept of truth; it was not trained on it. if we trained a ChatGPT inside of the Welcome to Night Vale universe it would give quite different answers.

* there is the extra strong condition of not causing bad PR for OpenAI

[f6v]

> It's true that when it reasons about the world ChatGPT

Does it though?