[JKCalhoun]

Perhaps then the solution is that LLMs need to be aware when the chat crosses a threshold and becomes talk of suicide.

When I was getting my Education degree, we were told that, as teachers, to take talk of suicide by students extremely seriously. If a student talks about suicide, a professional supposedly asks, "Do you know how you're going to do it?" If there is an affirmative response, the danger is real.

I suspect that comes from examining case studies?