[hammyhavoc]

Yes, which is the point: if you substitute a human therapist for an LLM, this is what's going to happen because "guard rails" can't account for every scenario. If half a billion dollars doesn't buy a bug-free game, why would it buy a safe LLM?

Does suicide encouraged by a therapist happen with human therapists too? Probably, but likely much less common as a suicide is going to hurt your reputation.