[salamo]

I think Ezra Klein framed it right during his interview of Gary Marcus:

> And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the truth.

ChatGPT isn't quite lying because it doesn't know what the truth is in the first place.

https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...

[bambax]

Yes and no. It's true that when it reasons about the world ChatGPT doesn't know the truth and doesn't care, and just aims to be plausible, and it's simply bullshitting. And yes, this is the main problem with ChatGPT.

But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying.

It's also a weird personality trait, something that is unlikely to have emerged on its own but was probably programmed into it. I don't know that for sure of course and have no information about how ChatGPT was put together, but from the outside it's troubling.

[salamo]

ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one.

This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely.

No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.)

This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though.

[1] https://arxiv.org/abs/2104.07567

[afiori]

It is indeed possible that enough of its training material contains enough absurdities so that conversations like:

> is X true?

> of course, X is true.

> I did not expect you to say that X is true.

> I never said that X is true, it must be false.

to have it methodically lie.

Personally I believe that a rolling bullshit generator is more plausible; ChatGPT and GTP3 before it worked by producing plausible authoritative* answers; in its training it found out that saying true things is an efficient way not to mess up.

But otherwise it has no concept of truth; it was not trained on it. if we trained a ChatGPT inside of the Welcome to Night Vale universe it would give quite different answers.

* there is the extra strong condition of not causing bad PR for OpenAI

[f6v]

> It's true that when it reasons about the world ChatGPT

Does it though?

[astrange]

This isn’t true, though. ChatGPT answers are certainly not always true (like here) but they are correlated with the truth, not “no relationship”.

Gary Marcus is basically an anti-AI crank who only ever says bad things about modern research and wants to go back to the old “expert systems” that didn’t work. Ezra Klein is, well, too invested in his Ezra Klein persona (a sort of extremely nice, slow speaking, therapeutic Mr. Rogers for people who read opinion columnists) to have time left to combat guest bias.

In this case the AI is unable to count letters because we humans don’t let it learn what letters are, as we trapped it in a computer and only let it experience byte-pair encoding tokens.

[robgibbons]

Reminds me of a quote I first heard in Lawrence of Arabia, "The man who tells lies hides the truth, but the man who tells half-lies has forgotten where he put it." (Robert Oxton Bolt)

[schwartzworld]

As George Costanza so eloquently put it: it's not a lie if you believe it.

[tialaramex]

Yes, ChatGPT is even more the essence of a bullshitter than Donald Trump was. Back when they were both leaders I would say that the crucial difference between Trump and Boris (Johnson, the UK Prime Minister at the end of Trump's period in office) was that Boris is a liar whereas Trump is a bullshitter, and for humans I think they were pretty poles apart in this respect. But ChatGPT is not burdened with the pragmatic knowledge of its immediate world that constrains even Donald Trump. When Trump denies something you can both see for yourselves is true, that's not bullshit, it's just lying. When ChatGPT says this fifteen letter "word" has sixteen letters it honestly has no idea that's not true.