[komodo]

There is a Chatbot AI product called character.ai that has suffered a marked decline in quality since its launch as they battle their users to maintain the AI’s safety protocols (similar to chatGPT “jailbreaks”). I wonder if something similar could be happening here.

[ecshafer]

These fighting against people using their product in “unauthorized” ways by the ai companies doesn’t make any sense to me. Who cares if character.ai users do some weird stuff with it, or replika creates romantic relationships, or people make off color jokes in gpt. There seems to be a lot of engineering effort driven by some product managers to have the ai do very specific things which makes the product much worse in the Long term.

[philovivero]

First, I am also frustrated by companies trying to prevent unauthorised used.

But second, the reasons are:

(1) For AI company, someone publishing: "I asked the model a question about crime, and it talked shit about black people! Look! [damning quote that you can also get model to say/do]." Stability took the "let people do what they will" tack and now Forbes and every other major media mouthpiece slams them at every opportunity about how they are ethically-challenged.

(2) For Replika, someone chatting with their online girlfriend: "I love you more than my wife and children." Then someone hacking Replika exposing these conversations, and now Replika is in hot water because all these divorces. Replace example with 100 other similarly awful situations like talking about mental health problems, crimes, petty squabbles with their coworkers, or political problems.

[zb3]

Forbes can write that crap, but the problem is with people who make decisions basing on that. I wonder who are these people that care about all this nonsense.

[happycube]

The problem is 'weird stuff' on chai included suicide, which sadly showed it needed better/more guardrails.