[senko]

The right way to deal with this is education.

1. We should be educating the populace about failure modes of AI chatbots - something well known to HNers but not to the general public.

2. CEOs, pundits, and marketers should be held accountable (morally, in the court of public opinion and in the news) when they present AI as unvarnished good - that's false marketing at the very least, and leads to tragic consequences.

3. The cat's out of the bag. With so many AI providers and models to choose from, and trivially-ran local models, you can't possibly police them all.

4. Instead of trying to shield the users (a futile task), we should educate them and equip them with knowledge on how to safely use the tools.

5. I'm cynical enough to believe governments all over the world will use the "think of the children" to institute de facto and/or de jure censorship and spying, pointing to the inability of the industry to treat the matter with actual respect.

[echelon_musk]

Educated consumers are harder to exploit and to convince to consume products that they might not need or want. It's antithetical to business and profit.

I'm cynical too.

Most peoples use of these tools is predicated in not knowing their flaws. If they did, I wonder if they would use the products as much?

[pants2]

What do you envision in the form of education? "Pass this test to use ChatGPT"?

[senko]

As in informing the public, not adding to the school curriculum.

A sustained collaborative effort by the AI makers, (local) governments and journalists could work, IMHO.

Instead AI makers hype it up, governments have no clue wtf is going on and media just chases clicks.

[johncolanduoni]

I don’t think education is going to help much when a person is experiencing full-blown psychosis. It’s not as if mental health professionals don’t already try to give their patients tools to counter delusional thought patterns, there is just a limit to what knowledge of one’s condition can do once their thinking becomes that disordered.

[senko]

I agree, but it might prevent the person entering that state due to encouragement from the AI?

We don't require licenses for knives because a psychotic person may injure themselves or others.

[johncolanduoni]

At least the cases that I’ve seen in the news have been of the form “person with established mental health issues enters feedback loop with sycophantic AI”. There may be cases that don’t fit that bill, but I haven’t seen them make headlines yet.

It’s also worth noting I don’t think we need a license or a ton of surveillance here. I think we can do a better job of moderating AI output to catch the AI telling people their family is plotting to murder them, and then send them a crisis hotline number instead. Sort of like what search engines do when you start googling methods of self-harm.