[noduerme]

I actually wish I could upvote this more. It's a great point.

Yes, if the LLMs didn't amplify whatever people already thought and feed it back to them as sycophantic praise, and instead scolded them into realizing they were doing something dumb, then maybe we would be having a totally different conversation.

But then the conversation might be about an LLM scolding someone into committing suicide instead of helping them commit suicide.

Both might be just as bad.