[barbazoo]

> But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.

It's not just when a recipient reports them it seems but also when they have been flagged by their algorithm. If that were true, the claim that the conversation is e2e encrypted simply cannot be true, unless the algorithm runs on the client.

[KaiserPro]

just think about the sheer number of people that report stuff.

given that facebook has less than 1k moderators, do you honestly think that they'd just let the moderators sift through everything manually?

obviously you'd classify stuff first, checking against known images is easy. Classifying new images is a lot harder, plus the ethics of training and labelling a dataset for accurate detection is pretty hard, also almost impossible to do legally.

I suspect the next best thing is detecting nudity and age of the subject, and taking the hit that you're going to prioritise a lot of malicious reports, rather than genuine.

[whoisjuan]

It sounds to me that there’s actually an algorithm between the report and the moderator to control the volume of manual moderation.

What you’re describing doesn’t work with E2E encryption. I really doubt it works that way.

[tg180]

>What you’re describing doesn’t work with E2E encryption

This smells like message franking, but I can’t be sure.

https://eprint.iacr.org/2017/664.pdf

[spywaregorilla]

Is there any particular reason to believe its not running on the client?

[KaiserPro]

because that would mean running a fairly large model on underpowered hardware. Also it would mean that you could never actually trust the output.

its far far more simple to run it server side on the reported message.