[onepunchedman]

I meant * could *. My point is that social engineering is a clear weak link in this system. They can also be sent regular photos whose hash matches the database, or use this repo to transform a regular pornographic photo's hash, making it hard for manual confirmation on Apple's part.

[robertoandred]

What kind of social engineering would lead an innocent person to save known CSAM to their photo library?

[copperx]

None needed. You could just send a photo to the target through WhatsApp, and the photo would be automatically synced with iCloud.

[simondotau]

Wouldn't the photo be scanned for CSAM by WhatsApp first?

[sethgecko]

Whatsapp messages are e2e encrypted so no.

[heavyset_go]

What kind of social engineering would lead an innocent person to install malware on their devices? Or do you think people like that want to take part in an illegal DDoS botnet?

[robertoandred]

I think there’s a difference between “I’ll click this totally legit button to protect my computer from viruses” and “I’ll save this picture of a child being raped to my photo library.”

A lot of people may not know how to avoid malware. But I don’t think very many of them would be so inept as to accidentally long press on child porn and tap “Add to Photos”.

[philipswood]

... and "I'll save this picture of an hilarious kitten to my photo collection"...

Fixed it for you.

The image to be saved doesn't have to be disturbing at all to trigger a hash collision.

The linked repo has code to modify an image to generate a hash collision with another unrelated image.

That's the whole point.

[Dylan16807]

If some commenters can be believed about their experience with the database, there are a bunch of completely innocuous images in it because they're from the same photosets or distributed alongside CSAM.

Is that enough to cause an investigation? Maybe, maybe not, but I wouldn't want it to be a risk.

[simondotau]

Photos in the database are classified for their content. Only images classified as A1 (A: prepubescent minor, 1: sex act) are being included in the hash set on iOS. So this doesn't even include A2 (2: lascivious exhibition), B1 or B2 (B: pubescent minor) let alone images which are in the database and aren't classified as any of A1, A2, B1 or B2.

While I've no doubt that there's a lot of "before and after" images (which are still technically CSAM even if they're not strictly child porn) and possibly many innocuous images, they would not have been flagged as "A1".

I'm sure there's probably still a few images flagged as A1 which shouldn't be in the database at all, but that number is going to be small. How many of these incorrectly flagged images are going to make their way into your photo library? One? Two?

You need 30 in order for your account to be flagged.

[Dylan16807]

If someone is deliberately targeting you with them, 30 isn't very hard to reach.

[simondotau]

I think it’s implausible that someone can become aware of 30 images which are miscategorised as A1 CSAM. How would this malicious entity discover them? What’s the likelihood that this random array of odd images could make it into a target‘s photo library?

And what’s the likelihood that a human reviewer will see these 30 odd images and press the “yep it’s CSAM” button?

More likely as soon as Apple’s human review sees these oddball images, they’re going to investigate, mark those hashes as invalid, then contact their upstream data supplier who will fix their data and now those implausible images are now useless.

[onepunchedman]

Lending your phone to someone for a call, then a quick airdrop. Legitimate-looking emails with buttons. There's probably a list somewhere of proven attack vectors.