[ectopod]

There seem to be two things happening.

When a user reports a post it is (unsurprisingly) forwarded to the moderators.

Additionally, there is some kind of AI CSAM detector, which automatically forwards posts.

In both cases, it also forwards the previous five messages from the thread to the moderators.

[Jtsummers]

> Seated at computers in pods organized by work assignments, these hourly workers use special Facebook software to sift through streams of private messages, images and videos that have been reported by WhatsApp users as improper and then screened by the company’s artificial intelligence systems.

> Instead, WhatsApp reviewers gain access to private content when users hit the “report” button on the app, identifying a message as allegedly violating the platform’s terms of service. This forwards five messages — the allegedly offending one along with the four previous ones in the exchange, including any images or videos — to WhatsApp in unscrambled form, according to former WhatsApp engineers and moderators. Automated systems then feed these tickets into “reactive” queues for contract workers to assess.

From the actual ProPublica report. If their published understanding is correct, E2EE is not broken, but rather end users who are one of the ends of E2EE are sending the decrypted content to be moderated. The AI bit is a filter to reduce the amount of content passed on to human moderators.

From near that second quote:

> Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive.

That part is AI driven, but my reading is that the moderators do not get access to the encrypted data (the actual messages) only the behavior patterns, and from that make a determination of what to do.

[barbazoo]

Correct me if I'm wrong but unless the "AI CSAM detector" is running on the client, it simply cannot be e2e encrypted.

[roywiggins]

It sounds like the only unencrypted data that the moderators see is sent from an endpoint (a user clicking "report"). After that an AI looks at the report and prioritizes ones that looks like it might be CSAM.

[ectopod]

Yes, so I assumed it is running on the client, but for all I know they could be encrypting the message and sending an image hash to Facebook.

[spullara]

It looks like the AI stuff applies to the groups content which is not E2E.