[einpoklum]

> This is not true, and it would be a very dangerous place to snorkel there.

For some reason people have the idea that truth is something ChatGPT optimizes for. Or safety of its conversant. That is absolutely not the case. IIUC, it optimizes for its answers sounding like an answer someone might give in a conversation (or on a web page or whatever). That often coincides with truth and safety, but - no more than that.

But its confidence was scary.