ChatGPT is assertive, authoritative, and even goes on to lie to justify its answers and "be right". It's a gasslighting machine. Here's an example I posted here a month ago:
I think Ezra Klein framed it right during his interview of Gary Marcus:
> And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the truth.
ChatGPT isn't quite lying because it doesn't know what the truth is in the first place.
Yes and no. It's true that when it reasons about the world ChatGPT doesn't know the truth and doesn't care, and just aims to be plausible, and it's simply bullshitting. And yes, this is the main problem with ChatGPT.
But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying.
It's also a weird personality trait, something that is unlikely to have emerged on its own but was probably programmed into it. I don't know that for sure of course and have no information about how ChatGPT was put together, but from the outside it's troubling.
ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one.
This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely.
No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.)
This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though.
It is indeed possible that enough of its training material contains enough absurdities so that conversations like:
> is X true?
> of course, X is true.
> I did not expect you to say that X is true.
> I never said that X is true, it must be false.
to have it methodically lie.
Personally I believe that a rolling bullshit generator is more plausible; ChatGPT and GTP3 before it worked by producing plausible authoritative* answers; in its training it found out that saying true things is an efficient way not to mess up.
But otherwise it has no concept of truth; it was not trained on it. if we trained a ChatGPT inside of the Welcome to Night Vale universe it would give quite different answers.
* there is the extra strong condition of not causing bad PR for OpenAI
This isn’t true, though. ChatGPT answers are certainly not always true (like here) but they are correlated with the truth, not “no relationship”.
Gary Marcus is basically an anti-AI crank who only ever says bad things about modern research and wants to go back to the old “expert systems” that didn’t work. Ezra Klein is, well, too invested in his Ezra Klein persona (a sort of extremely nice, slow speaking, therapeutic Mr. Rogers for people who read opinion columnists) to have time left to combat guest bias.
In this case the AI is unable to count letters because we humans don’t let it learn what letters are, as we trapped it in a computer and only let it experience byte-pair encoding tokens.
Reminds me of a quote I first heard in Lawrence of Arabia, "The man who tells lies hides the truth, but the man who tells half-lies has forgotten where he put it." (Robert Oxton Bolt)
Yes, ChatGPT is even more the essence of a bullshitter than Donald Trump was. Back when they were both leaders I would say that the crucial difference between Trump and Boris (Johnson, the UK Prime Minister at the end of Trump's period in office) was that Boris is a liar whereas Trump is a bullshitter, and for humans I think they were pretty poles apart in this respect. But ChatGPT is not burdened with the pragmatic knowledge of its immediate world that constrains even Donald Trump. When Trump denies something you can both see for yourselves is true, that's not bullshit, it's just lying. When ChatGPT says this fifteen letter "word" has sixteen letters it honestly has no idea that's not true.
ChatGPT is not human, and cannot be more assertive, authoritative or lying than Python.
If you give python "0.1 + 0.3", it will instantly, and without a doubt, output "0.30000000000000004". Is Python arrogantly thinking it's right and misleading you?
Of course not. This last sentence makes no sense, Python is just a program that does what it's programmed to do.
But because ChatGPT is using natural language as a UI, it triggers a very irrational response in humans interacting with it.
We are starting to anthropomorphize it, use adjective like "assertive", "authoritative", and "lie" to label its output, which is like calling Covid cruel and vicious because it kills old people, or a NPC lazy because it does not work to pay for its food.
There is no intent of ChatGPT to deceive or justify its supposed lies. There is only a program that does what's its programmed to do, and shows its limitations.
What do you expect, that is says "I think it's x but I can be wrong with a confidence interval of z" with every answer? Do you expect google to tell that when it outputs the results?
GPT is not talking to you. It's an output. It's not confident. It's a program printing text matching statistical analysis.
It's worrying that such a simple program is already close enough to the real thing that people, even on HN, are already attributing it character.
It's true that ChatGPT is just a program, a machine, and that we are "starting to anthropomorphize" it. But that's what it wants! (Or, what its makers want.)
> Do you expect google to tell that when it outputs the results?
Well, Google doesn't tell me anything, it quotes things it has found on the web.
The huge, huge difference between Google and ChatGPT is that Google never, ever says "I", while ChatGPT does. The other difference is that Google produces references that are actual links to actual web pages it has not authored, while ChatGPT speaks in its own name and when asked to produce references, simply makes them up (lots of examples about this).
If you can't see how ChatGPT is totally different from a regular search engine in its positioning and proposed usage, you're not alone, and I'm afraid I can't help you. But -- IMHO -- you are very very wrong.
I can see that it's totally different from a search engine. If you can't understand that analogies don't need (and cannot be) to be perfect to be useful, you're not alone, and I'm afraid I can't help you. But -- IMHO -- you are very very wrong.
> And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the truth.
ChatGPT isn't quite lying because it doesn't know what the truth is in the first place.
https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...