[mediaman]

People like us are not the real users.

Corporate users of AI (and this is where the money is) do want safe models with heavy guardrails.

No corporate AI initiative is going to use an LLM that will say anything if prompted.

[adam_arthur]

And the end users of those models will be (mostly) frustrated by safety guardrails, thus perceive the model as worse and rank it lower.

[baobabKoodaa]

Yep. And in addition, lobotomized models will perform worse on tasks where they are intended to perform well.