[lazide]

Just what we need, the LLM ‘Reddit cares’. Notably, that is a mess too.

Also plenty of those hotlines are BS, or don’t work, or flat out don’t exist for given locales, etc.

The biggest issue is that LLM’s can act like a person, but aren’t a person, and fundamentally this causes problems. Especially for people that are already borderline or fully crazy.

[ACCount37]

The biggest issue is that LLMs can act like a person indeed. And the kind of person an LLM acts as? Well.

When you train on raw user feedback, you can easily end up wiring some incredibly undesirable patterns into your AI. Resulting in things like an AI that never wants to contradict its user, and always wants to support its user in everything, and always wants the user to like it. See GPT-4o for the kind of outcomes that results in.