[_k7dr]

The chatbot in that article did have safeguards in place to stop it from listing suicide methods, but he just kept asking until he found a jailbreak. He was looking for someone to give him the answer he wanted all along

Past a certain point is there anything that will stop someone that determined to find something to validate their own self-destructive viewpoint? If not AI then a site or faction or person on Telegram with a pro-suicide opinion could have the same effect, and it would still be just as tragic but no longer be noteworthy. It didn't sound like he had a happy life to begin with with the AI ruining it completely, like it sounds

[hammyhavoc]

I had an LLM tell me to that killing myself makes sense when recounting a low point in my life to it and how I'd contemplated suicide. I wasn't looking for any kind of "jailbreak" in prompts, this was following a genius on HN saying it's a "great alternative to a therapist". No, it isn't fit for purpose.

> It didn't sound like he had a happy life to begin with

Which are exactly the kind of people who are going to be using LLMs as a "therapist". See the problem?

[d0mine]

LLMs make mistakes quite often. It is unreasonable to expect that it stops making mistakes just because suicide is mentioned.

LLM is a tool. There are many tasks it is good at and even more tasks where it sucks e.g., GPT-4 can fail on trivial chess questions.

Avoid permanent solutions to temporary problems.