[dissahc]

I think the presence of grammatical errors and other idiosyncrasies is a strong indicator that they aren't confabulated.

[krainboltgreene]

Why? Are not grammatical errors and idiosyncrasies part of the data set that chatGPT is trained on and thus also a predictable set of tokens?

[btilly]

Such errors are rare enough that they are almost never the most likely next token. Therefore ChatGPT only makes such mistakes when it is given good reason to do so.

[krainboltgreene]

ChatGPT cannot reason.

[SirMaster]

How do we know they aren’t fake system prompts planted there by OpenAI to be returned to the user when people try to get them?

[datameta]

General System's RetroConfabulator