[cookiengineer]

Imagine being a parent that made pictures of their own children that bathed naked in their own backyard.

I don't know about you, but my parents certainly have lots of embarassing pictures of me in their photo album.

There will be so many false positives in that system, it's ridiculous. It doesn't necessarily have to be a false colliding hash, but legitimate use cases that - by definition - are impossible to train neural nets on unless the data is being used illegally by Apple.

[ec109685]

That’s not how Apple’s system works. It’s not an image classifier. Only actual images that are derivatives of known CSAM images (a database of 250k images) will match. Random images of kids will not match those at any greater frequency than any other image.

[cookiengineer]

Counter-question: At what point is child porn actually child porn, socially and statistically speaking?

If I share that picture of my child with my friends and loved ones on Facebook - at what "scale" is it considered to be added to that database as child porn?

1k shares? 10k? Who's the one eligible to decide that? The judicatives? I think this scenario is a constitutional crisis because there's no good solution to it in terms of law and order.

[jazzyjackson]

I think you're underestimating the severity of child abuse by orders of magnitude. CSAM is a database of child rape, not child nudity.

[foepys]

For now. You don't know what will be added next.

China will demand it to include pictures of the Tiananmen massacre.

[jazzyjackson]

Well that’s a pretty orthogonal concern to the above comment that was worried about getting flagged for sharing pics of their own kids

[noduerme]

idk what's in the database, whether it's rape or nudes or both. Although depictions of sexual acts versus simple nudity seems like a logical place to draw a line, all the lines on adult pornography are arbitrarily drawn based on "community standards", and we're only a few decades away from state-level bans on any nudes as "porn" in the US, including artistic photos. (Not to mention anti-sodomy laws).

Even if what's in the database is 100% violently criminal as you suggest, and even if it remains limited to that material, we already have a process in place that denies the accused of even seeing the evidence against them if a hash matches. What a horrific, orwellian situation if someone sent you hash matches, the police raid your house and now you can't even see what they think they have or prove your own innocence.

[ec109685]

You would presumably have the 30+ images on your device or in iCloud to prove your innocence.

For you to get caught up in this dragnet, 30+ plus images have to match NeuralHash’s of known illegal images, thumbnails of those images have to also produce a hit when run through a private hash function that Apple only has, and two levels of reviewers have to confirm the match as well.

[onepunchedman]

What does Apple even do in this situation? That media won't match known CSAM, but if you modify childhood images so that its hash matches CSAM, what does Apple do. There are just SO MANY things that can and will go wrong as people try to exploit this system.

[zepto]

You can’t modify your childhood images so their hash matches csam because the visual derivative won’t match.

[ummonk]

In the digital age, I certainly wouldn't be taking such pictures, let alone uploading them to cloud storage. Not because of any concerns about neural hashing, but simply because I wouldn't want such pictures of my children getting leaked / stolen / hacked.