[proto-n]

No, what you are writing is not correct. It can make up "information" on the fly. It's goal is to sound plausible, not to tell the truth.

Second, the "language model" is a text completion engine. What can be controlled using conversational natur language is ChatGPT, which is a conversational engine developed out of a language model.

[lwhi]

I think you're grossly overestimating what is happening.

Why exactly have you placed information between quote marks?

[proto-n]

I don't think I am: I just read the transcript of a quite funny chatgpt conversation, where it was asked to explain about a hungarian poet (Petőfi Sándor) in hungarian. It listed some true facts, then a list of the "most famous poems", including a bunch that didn't exist. Then the user asked about a particular one titled "The Panther". It responded by explainin about the poem, how it's main theme is fight for freedom (Petőfi was a big figure in the hungarian revolution of 1848), and how the panther itself symbolizes freedom and self-sacrifice. It even quoted a few lines from the poem (it didn't rhyme, but otherwise was very plausible to have been written by Petőfi).

See, "The Panther" is not a poem of Petőfi. The quote was not from any other work of him. It's not from the Rilke poem (or its translations I know of) either. It's completely made up, but also made to sound very plausible, to the point where if I didn't know better or look it up, I could be convinced. The only thing suspicios is the lack of rhyming.

I put information in quotes because I don't consider made up stuff information.

[lwhi]

How are you coming to the conclusion that it's invented information, rather than — reworded / rephrased — incorrect information that an actual human has provided online?

[proto-n]

I can't exclude the possibility. But kind of in the same sense that I can't exclude the possibility that it's actually true. Sure, might be that I'm simply not aware of the poem, etc. I find this extremely unlikely.

I also find it extremely unlikely that someone on the internet invented this tale about "the panther" and chatgpt just rephrased it or quoted it. The internet is full of actual true lists of Petőfi's famous poems, but it isn't full of people inventing fake poems of his.

On a very theoretical level, sure, it's a rephrasing and combination of pieces of stuff ChatGPT has actually been trained on, because it has seen hungarian text, it has learned stuff about Petőfi, it has seen every single word that its using. But after a certain point, combining known words and text structures with bits of semantic knowledge in unexpected ways becomes a new invented thing, instead of just quotes.

But you can see many examples of ChatGPT inventing things like urls, python libraries, etc. It's perfectly capable of bulshitting believably.

[lwhi]

> I can't exclude the possibility. But kind of in the same sense that I can't exclude the possibility that it's actually true. Sure, might be that I'm simply not aware of the poem, etc. I find this extremely unlikely.

What you're describing is a hunch.