[ec109685]

That’s not how Apple’s system works. It’s not an image classifier. Only actual images that are derivatives of known CSAM images (a database of 250k images) will match. Random images of kids will not match those at any greater frequency than any other image.

[cookiengineer]

Counter-question: At what point is child porn actually child porn, socially and statistically speaking?

If I share that picture of my child with my friends and loved ones on Facebook - at what "scale" is it considered to be added to that database as child porn?

1k shares? 10k? Who's the one eligible to decide that? The judicatives? I think this scenario is a constitutional crisis because there's no good solution to it in terms of law and order.

[jazzyjackson]

I think you're underestimating the severity of child abuse by orders of magnitude. CSAM is a database of child rape, not child nudity.

[foepys]

For now. You don't know what will be added next.

China will demand it to include pictures of the Tiananmen massacre.

[jazzyjackson]

Well that’s a pretty orthogonal concern to the above comment that was worried about getting flagged for sharing pics of their own kids

[noduerme]

idk what's in the database, whether it's rape or nudes or both. Although depictions of sexual acts versus simple nudity seems like a logical place to draw a line, all the lines on adult pornography are arbitrarily drawn based on "community standards", and we're only a few decades away from state-level bans on any nudes as "porn" in the US, including artistic photos. (Not to mention anti-sodomy laws).

Even if what's in the database is 100% violently criminal as you suggest, and even if it remains limited to that material, we already have a process in place that denies the accused of even seeing the evidence against them if a hash matches. What a horrific, orwellian situation if someone sent you hash matches, the police raid your house and now you can't even see what they think they have or prove your own innocence.

[ec109685]

You would presumably have the 30+ images on your device or in iCloud to prove your innocence.

For you to get caught up in this dragnet, 30+ plus images have to match NeuralHash’s of known illegal images, thumbnails of those images have to also produce a hit when run through a private hash function that Apple only has, and two levels of reviewers have to confirm the match as well.