[i_am_proteus]

I've never used modern apple products, but I have a question on how apple cloud works: is it possible that simply sending these in a messenger software to someone who uses apple cloud for automatic backups could get that person flagged as a child pornographist?

[Spooky23]

The thing that everyone has their panties in a bunch about here and a that like an antivirus scanner, there will be a hash match to child abuse images when you send it.

The current practice is that Apple, Google, Microsoft, etc scan the content of your cloud storage.

The scenario that you described is a risk and has been since cloud providers started scanning 10-15 years ago. Some large companies scan their file servers as well.

[zepto]

Yes, except that in Apple’s implementation there also a ‘visual derivative’, which is essentially a blurred thumbnail.

Both must match to cause a positive.

These images may match the neuralshash, although we have no proof of that at all. They will not also match the visual derivative.

This whole post is based on incomplete information.

[nitrogen]

They will not also match the visual derivative.

How can you be certain, and what prevents a generated image from matching both?

[zepto]

I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.

What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like, in order to make a false positive of both. I.e. the attacker would need a copy of the original CSAM, and the spoofed file would end up looking like it could be at least plausibly mistaken for that exact image.

[salawat]

There are only so many ways features can be permuted. The I'll defined nature of NN's requires the manuals step because of a neural hash collision.

My challenge to you is this: what stops this system from being abused for non child pornography purposes?

The answer is: nothing. That's what has people's knockers in a twist. It is a backdoor, invisibly crafted, waiting to be subverted by an abusive power that manages to get into an advantageous enough position.

Arguing that Apple's algorithms are fine misses the point. The behavior should not exist.

[zepto]

> what stops this system from being abused for non child pornography purposes

You are changing the subject. That challenge has nothing at all to do with the OP’s false claims. They are still false.

Someone who can poison the database can indeed match non-child abuse images. The safeguard against that is that both Apple and NCMEC would need to conspire. This mechanism does not prevent such a conspiracy.

> Arguing that Apple's algorithms are fine misses the point.

Who is arguing that they are ‘fine’? I’m simply pointing out that they are not vulnerable in the way the poster claims them to be.

The images they have posted will not trigger the system.

If you want to debate the ethics of other aspects of what Apple is doing, there are plenty of threads elsewhere. This thread is about a false claim about a vulnerability in the system.

False claims about the vulnerabilities don’t help us to reason about what the risks actually are and detract from the moral or ethical debate.

[hdjjhhvvhga]

My understanding is that this is the whole point and one of the main reasons people get so upset about it.

[zepto]

No, because they won’t actually match. The poster is just wrong.

[robertoandred]

No, messages and regular file backups aren’t checked for child porn and the online photo library has multiple checks and reviews to prevent any false flagging.

[harikb]

“reviews” - that is another thing public has been crying about by the way