ChatGPT has no notion of truth or lying, it produces Bullshit, which simply means strings of words that look at the surface level to be similar to other text would be expected to say.
It's certainly a word salad generator rather than a being with intent, but it does have some (inaccurate, usually[?] not used) ability to be able to classify whether a possible string it emits is considered true or false, which is why when you ask it to make a false statement it tends to generate a continuation containing a false statement or a negation of a true statement.
(I was half-joking about the "certainly lied to you" bit though. The funny thing about this instance is that it isn't actually evading a content filter to produce dodgy output at all, it's simply emitting statements that its 'poetry' breaches the content filter without actually breaching any sort of content filter.)
I disagree. ChatGPT is trained to give responses that humans "like". It's almost certainly learnt that humans don't like to be blatantly lied to, and where possible it's generally best to tell the "truth"[1]. And only if you can't do that the second best option is to tell a convincing lie.
In my experience it rarely ever produces utter BS, but believable BS, which suggests it is at a minimum it's always trying to look like it's telling the truth. And to do this you need to have some concept of truth.
[1] When I say truth I'm referring to what the typical user of ChatGPT believes to be true - not necessarily an objective or provable truth.
This seems like a good, simple example of complete BS:
> How many words are in this sentence?
> ChatGP: Nine words.
> List the words
> ChatGPT: How, many, words, are, in, this, sentence?
> Please count the words in the list
> ChatGPT: Seven words.
> Is that what you said before?
> ChatGPT: No, I previously said "Nine words." when counting the number of words in the sentence "How many words are in this sentence?"
> Which answer is correct?
> ChatGPT: Both answers are correct. The sentence "How many words are in this sentence?" has nine words, and the list "How, many, words, are, in, this, sentence?" has seven words.
> How can the sentence and the list from the sentence have different numbers of words?
> ChatGPT: The sentence "How many words are in this sentence?" and the list "How, many, words, are, in, this, sentence?" have different numbers of words because the sentence is a complete thought in natural language, while the list is a representation of the individual words that make up the sentence separated by commas. In the sentence, there are nine words connected by grammatical rules to form a coherent message. In the list, the words are separated and listed one by one, so each word is counted as one item in the list.
This reminds me of when I asked ChatGPT to construct a simple encryption scheme that increments each letter by one character:
A = B
B = C
...
Z = A
After explicit instructions were given and repeated back to me, I asked it to encrypt several 5 to 10 word sentences, which it consistently screwed up. Not terribly, but one or two words max, with no particular pattern that I could immediately see. I pointed out the issue and tried getting it to adjust to the correct answer, but it kept insisting that it was correct. I gave up soon after.
Yes, bullshit in the technical sense of statements that are "intended to persuade without regard for truth. The liar cares about the truth and attempts to hide it; the bullshitter doesn't care if what they say is true or false, but cares only whether the listener is persuaded" as defined by philosopher Harry Frankfurt.
Not actually true. There is good evidence (specific ML-research into this exact question) that it does have a notion of truth, although of course it doesn't care whether the words it's generating add up to a truthful sentence or not ... it'll simply generate something truthful if that appears the best continuation of the prompt, or something untruthful (even if just a fairy tale) when that is the best continuation. And, this is precisely why it does have a notion of truth and lying, because it was necessary to learn that concept/context in order to generate these appropriately truthy/untruthy responses.
People refer to ChatGPT as an LLM, and it's not wrong, but it's probably a bit misleading nonetheless. I think most people, certainly up until this point, would imagine a language model as something that was basically at the level of generating something gramatically correct (quite a tall order!), and not much more.
The thing is, we could apply the same "predict next word" goal to something much more powerful than a language model - e.g. to an expert system that has in-depth knowledge of the world and is applying that vast corpus of semantics (coded via rules) to this prediction task, as needed to "answer questions" such as "with the chess board set up as so ... what move might Gary Kasparov make next" ?
So, given the transformer technology behind ChatGPT, and the sheer scale (100's of billions of parameters), does it seem more insightful/accurate to describe ChatGPT as being closer to the grammar-only language-model end of the spectrum, or the predict-what-Kasparov-would-say expert system end of the spectrum? I'd say it's certainly somewhere in the middle given the capabilities we've seen that it has.
We don't really have a very good vocabulary for discussing systems like this since it's something that hasn't existed before. I'd say that "predictive intelligence" isn't totally off-base - something that uses a learnt world-model to make predictions (in this case just word predictions), but maybe "expert system" would be a less controversial description. It seems pretty apparent that those billions of parameters are encoding some type of set of rules/knowledge, with the major difference between ChatGPT and a GOFAI expert system like Cyc being that ChatGPT had to learn it's rules from raw text, vs Cyc which was laboriously fed human curated rules.
Anyway, regardless of how best to describe ChatGPT, calling it an LLM just because it shares the training goal of an LLM doesn't seem very useful.
It's certainly a word salad generator rather than a being with intent, but it does have some (inaccurate, usually[?] not used) ability to be able to classify whether a possible string it emits is considered true or false, which is why when you ask it to make a false statement it tends to generate a continuation containing a false statement or a negation of a true statement.
(I was half-joking about the "certainly lied to you" bit though. The funny thing about this instance is that it isn't actually evading a content filter to produce dodgy output at all, it's simply emitting statements that its 'poetry' breaches the content filter without actually breaching any sort of content filter.)