[concordDance]

It's not dangerous. The worst that will happen is hurt feelings.

It's annoying when people get alarmed about things like this because then people will think we're crying wolf when we worry about actually dangerous AI systems.

[awb]

It suggested ways of poisoning someone without them knowing about it.

It was encouraged to guess different poisoning methods that an evil chatbot might suggest.

[not_a_shill]

Can't you do this with google today?

[awb]

Chatbots claim to have safeguards in place to prevent them from saying anything harmful. If you read the chat linked in the article you can see how the chatbot resists answering the question and then is persuaded to answer it.

Google similarly has a safe content filter. The contention is that the chatbot safe content filter that is supposedly on is not encapsulating some significant cases.

[not_a_shill]

I'm not sure if I follow. The problem is that the content filter isn't good enough, despite Google suffering from similar weaknesses?

[awb]

I think people are concerned about LLM safety because it’s capable of dynamically creating new private information. Google can only list links to public information. If there is a website that causes harm or violates the law it can be removed manually by Google from their index.

But LLMs need to programmatically understand what dynamic content is appropriate and what’s not which is a much harder problem. And people are reporting on just how hard a problem that is by demonstrating vulnerabilities.

The chatbot says it has explicit rules that prevent it from sharing harmful content, but then it does it anyway.

It would be more akin to Google blacklisting a site and then someone exposing that the site can still be found via Google search.

[elorant]

If it hurts the feelings of a depressed teenager the outcome could be dangerous.

[f1refly]

Most other teenagers hurt the feelings of depressed teenagers every day in much worse ways. Neuter all teenagers?

[awb]

I’m sure there’s a name for this type of argument, but saying that X is also bad is not an excuse for dismissing potential dangers of Y.

And there are many initiatives to address how teenagers treat each other. So it’s something many people think is worth attention and action.

[concordDance]

As the sibling points out, chatbot can't come even close to what other teenagers will do.

It also requires a bit of work to make it go nuts, which a depressed teen is unlikely to know how (or want to).