This is not the nanny state, it's just PR concerns. They don't want to be associated with headlines like "FBI Finds Child Porn Images Generated by Microsoft's AI".
PR concerns about what puritans will think. Puritans love a nanny state with the entire modicum of affection their shrivelled black hearts can muster.
No, I'm not defending CSAM or AI simulacra of it. Fusing the medium with the censor is an abomination. It's Cocreator wagging the finger today, tomorrow it's Clippy popping up to tell me that what it thinks I might be making is too objectional for me to continue making it.
The difference probably is the hardware it runs on.
It feels like the same argument about hosting files: if you provide a service then you're responsible for what happens in it.
There's plenty of self hosted image generators without any guardrails whatsoever if you're so inclined. Browse around https://civitai.com/ for a bit you'll quickly see the implications
Please tell my why this prompt is blocked by Bing's image creator then: "view from inside a shark's mouth as it hunts fish" but "view from inside a shark's mouth as it looks for fish" is fine.
A shark hunting is over the line but it can look for fish fine?