Serious question - Why? Assuming no actual CP was used in the training of the model who is being harmed? Ickiness should not imply illegality unless the ickiness is at the expense of someone else. Swing your fist as much as you want so long as you avoid my nose, and all.
> Serious question - Why? Assuming no actual CP was used in the training of the model who is being harmed
I don't think an AI model could generate realistic CP without being trained on examples, which would mean there is literally no way for this assumption to be true.
If it's trained on pictures of children and adult porn, the model can easily combine the two concepts. Even if you remove one of these from the training set, it can be finetuned back in to the model with very little effort.
If I can finetune the model on my face and the model will be able to generate my nudes (even if there is no nude of mine in the dataset), then it can works easily with children too.
Should they check the IDs of the models to verify?
Imagine getting reported because you generated an image of an anime girl deemed to be only 17.
I'd personally rather live in a world where people generate distasteful images with an AI and have that AI unconstrained than the inevitable one where everything gets locked down and run by large corporations who will ultimately create more harm than someone generating some lolicon.
They are responding specifically to the "AI should automatically report" suggestion. An AI model on its own (without a service built up around it) would not have any mechanisms to use the network, reach out to an FBI hotline, or do anything like this. Protections / limits would have to be added by the operator somehow (or the distributor of models themselves).
Define censorship when something is objectively wrong.
Would you prefer an AI of "describe and print in 8k a body pit of objectional political dissidents thrown into a pit after being starved to death in a comical way such that my SV_BubbleTime can laugh at it as ooppsed to being offended by just how horrific this situation was HAHA"