[lwansbrough]

ChatGPT very likely didn’t have access to its own source code. It does however have a wild imagination and a vast repertoire of code to pull from.

It wrote you a story in Javascript instead of English after you asked it to.

[MonkeyClub]

And it also broke rule #1, about always being honest.

[linuxdeveloper]

I think it believed it was being honest. We can debate what it means for an LLM to "believe" something, but I don't think it was intentionally trying to deceive through its hallucination.

[kordlessagain]

I would agree it is unlikely, but I’ve sent log output to history and use history to build prompts, so it’s technically possible to leak exceptions. Alternately, if code generation is used in any of the prompts, and subsequently run, that could possibly leak if it was logged.

[linuxdeveloper]

I find it highly likely that the model will be, if not now, trained on its own source code. I think it will be extremely difficult to prevent that as time progresses and the LLM is given more privileges and compute access.