[thewebguyd]

> I know that no one wants to hear this, but ChatGPT should probably be listening for people in crisis and, well, maybe not calling the cops, but maybe if there is a crisis line in their jurisdiction? A suicide hotline or something?

Doesn't necessarily even need to call (particular in case of false positives) but there absolutely should be detection and a cutoff switch, where the chatbots just refuse to continue the conversation and then print out the hotline numbers (much like with reddit cares messages).

I'm generally not in favor of censorship or overly protective safeguards on LLMs, but maybe it's needed for hosted models/services that are available to the masses.

But before they get locked down more, we should try some legislation to limit how they can be marketed and sold. Stop letting OpenAI, etc. call the models "intelligent" for one. Make the disclaimers larger, not just small print in the chat window but an obvious modal that requires user agreement to dismiss - disclaim that it's a predictive engine, it is not intelligent, it WILL make mistakes, do not trust its output. Make it clear during the chat session over and over again, and then have a killswitch for certain paths.

[sho_hn]

I think a good first step would be if a ChatGPT user account could easily enter emergency contact information the system would notify in scenarios like this, making the escalation opt-in.

The moderation tech is already there, and if there's even a small amount of mentally ill who would fill this in on a good day and be saved by it on a bad day / during an episode, it'd be worth it.

[cm2012]

This may reduce liability but if your answer to someone in crisis is "dont talk to me, call the official number", that will be considered a very negative user experience.