|
|
|
|
|
|
by rtkwe
550 days ago
|
|
|
According to the complaint that comes from the kid's own interactions with the bot not some post hoc attempt to prompt engineer the bot into spitting out a particular response. The actual claim is linked in the article if you care to read it, it's not stating the app can produce these messages but that it did in their kid's interactions and C.Ai has some liability for failing to prevent it. |
|
|
For one, 99% of the "roleplay" models eventually drag into one of a handful of endgames: NSFW RP, suicide discussion, nonsensical rambling, or some failsafe "I don't know" state where it just slowly wanders into the weeds and directs the conversation randomly. This can be anywhere from a few messages in (1bq) to hundreds (4-6bq) and sometimes it just veers off the road into the ditch.
Second, the UIs for these things encourage a "keep pressing the button until the thing you want comes out" pattern, modeled off of OpenAI's ChatGPT interface allowing for branching dialogue. Don't like what it said? Keep pushing the button until it says what confirms your bias.
Third, I can convince most of the (cheaper) models to say anything I want without actually saying it. The models that Character.AI are using are lightweight ones with low bit quantization. This leads to them being more susceptible to persuasion and losing their memories -- Some of them can't even follow the instructions in their system prompt beyond the last few words at times.
Character.AI does have a series of filters in place to try and keep their models from spitting out some content (you have to be really eloquent at times and use a lot of euphemism to make it turn NSFW, for instance, and their filter does a pretty decent job keyword searching for "bad" words and phrases.)
I'm 50/50 on Australia's "16+ for social media" take but I'm quickly beginning to at least somewhat agree with it and its extension to things like this. Will it stop kids from lying? No. It's a speedbump at best, but speedbumps are there to derail the fastest flyers, not minor offences.