[hammyhavoc]

I had an LLM tell me to that killing myself makes sense when recounting a low point in my life to it and how I'd contemplated suicide. I wasn't looking for any kind of "jailbreak" in prompts, this was following a genius on HN saying it's a "great alternative to a therapist". No, it isn't fit for purpose.

> It didn't sound like he had a happy life to begin with

Which are exactly the kind of people who are going to be using LLMs as a "therapist". See the problem?

[d0mine]

LLMs make mistakes quite often. It is unreasonable to expect that it stops making mistakes just because suicide is mentioned.

LLM is a tool. There are many tasks it is good at and even more tasks where it sucks e.g., GPT-4 can fail on trivial chess questions.

Avoid permanent solutions to temporary problems.