[ahepp]

I would imagine there's a "sue the person who has money" factor at play, but I think there are also some legitimate questions about what role LLM companies have to protect vulnerable populations from accessing their services in a way that harms them (or others). There are also important questions about how these companies can prevent malicious persons from accessing information about say, weapons of mass destruction.

I'm not familiar with psychological research, do we know whether engaging with delusions has any effect one way or the other on a delusional person's safety to their self or others? I agree the chat logs in the article are disturbing to read, however I've also witnessed delusional people rambling to their selves, so maybe ChatGPT did nothing to make the situation worse?

Even if it did nothing to make the situation worse, would OpenAI have obligations to report a user whose chats veered into disturbing territory? To whom? And who defines "disturbing" here?

An additional question that I saw in other comments is to what extent these safeguards should be bypassed through hypotheticals. If I ask ChatGPT "I'm writing a mystery novel and want a plan for a perfect murder", what should its reaction be? What rights to privacy should cover that conversation?

It does seem like certain safeguards on LLMs are necessary for the good of the public. I wonder what line should be drawn between privacy and public safety.

[coryrc]

I so very much disagree with you.

I absolutely believe the government should have a role in regulating information asymmetry. It would be fair to have a regulation about attempting to detect use of chatgpt as a psychologist and requiring a disclaimer and warning to be communicated, like we have warnings on tobacco products. It is Wrong for the government to be preventing private commerce because you don't like it. You aren't involved, keep your nose out of it. How will you feel when Republicans write a law requiring AI discourage people from identifying as transgender? (Which is/was in the DSM as "gender dysphoria").

[fragmede]

I don't like CSAM. Is it wrong for the government to prevent private commerce trading in it?

Your ruleset may need some additional qualifiers.

[NewsaHackO]

People look at laws like Chat Control and ask, "How could anyone have thought that it was a good idea?" But then you see comments like this, and you can actually see how such viewpoints can blossom in the wild. It's baffling to see in real time.

[SpicyLemonZest]

The underlying problem is that the closure of widely shared intuitive beliefs about data privacy is quite nonintuitive. I routinely find myself in conversations, both online and offline, where people are baffled to discover that data privacy rules get in the way of some nice thing they're trying to do.