[breck]

How would such a thing work though?

Would it be a map of unacceptable things, or an on device model like GPT3? Would you allow someone to type the offending thing and just disable the send button, or would you block it at the keyup event?

I don't see any way this could be implemented without kissing freedom goodbye.

I like the other commenter's take that this idea was just a bit out of exasperation.

[xenonite]

Well, Apple just stated that they are one of the best in privacy oriented AI because their AI could run on-device. And I suppose the detection of such things would be a good example.

But I see your point: you would need to inform the user without restricting his or her freedom.