[Hugsbox]

Don't give AI the ability to unilaterally remove/reject comments. Allow AI to flag comments as potentially harmful, and then have a human review whether or not they should be removed. AI shouldn't have the agency to actually do anything, just the ability to say "Hey, this comment might be against the rules, you should have a look!"

[linmer]

I think approving the comment is also doing something, so if we won't allow it to do that too it will be useless. But I think you mean we shouldn't allow AI to do potentially harmful actions like removing/rejecting comments. But approving a bad comment also means people see it so it's a debate about which one being more harmful in practice. But I agree with you!

[JohnFen]

Don't use genAI to approve comments, either. Only use it to flag potentially problematic comments for human review. The harm of people temporarily seeing a problematic comment is far less than the harm of removing comments that shouldn't have been removed.

[gus_massa]

I'm not sure if it's relevant, but SpamBayes uses a not-so-naive Bayes classification and it has a ham(good) and spam(bad) points. IIRC the usual interface classify them by ham/(ham+spam) and the idea is that if the quotient is bellow 10% or over 90% then it's classified automatically, and in between it goes to a "interesting" bucle for human review. I used it a long time ago, I think I changed the defaults to 1% and 99% that is correct almost all the time and I took a look at the "interesting".

[linmer]

That's a great idea. And for deep nets you can do the same with probabilities. Thanks!