[Spooky23]

The thing that everyone has their panties in a bunch about here and a that like an antivirus scanner, there will be a hash match to child abuse images when you send it.

The current practice is that Apple, Google, Microsoft, etc scan the content of your cloud storage.

The scenario that you described is a risk and has been since cloud providers started scanning 10-15 years ago. Some large companies scan their file servers as well.

[zepto]

Yes, except that in Apple’s implementation there also a ‘visual derivative’, which is essentially a blurred thumbnail.

Both must match to cause a positive.

These images may match the neuralshash, although we have no proof of that at all. They will not also match the visual derivative.

This whole post is based on incomplete information.

[nitrogen]

They will not also match the visual derivative.

How can you be certain, and what prevents a generated image from matching both?

[zepto]

I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.

What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like, in order to make a false positive of both. I.e. the attacker would need a copy of the original CSAM, and the spoofed file would end up looking like it could be at least plausibly mistaken for that exact image.

[salawat]

There are only so many ways features can be permuted. The I'll defined nature of NN's requires the manuals step because of a neural hash collision.

My challenge to you is this: what stops this system from being abused for non child pornography purposes?

The answer is: nothing. That's what has people's knockers in a twist. It is a backdoor, invisibly crafted, waiting to be subverted by an abusive power that manages to get into an advantageous enough position.

Arguing that Apple's algorithms are fine misses the point. The behavior should not exist.

[zepto]

> what stops this system from being abused for non child pornography purposes

You are changing the subject. That challenge has nothing at all to do with the OP’s false claims. They are still false.

Someone who can poison the database can indeed match non-child abuse images. The safeguard against that is that both Apple and NCMEC would need to conspire. This mechanism does not prevent such a conspiracy.

> Arguing that Apple's algorithms are fine misses the point.

Who is arguing that they are ‘fine’? I’m simply pointing out that they are not vulnerable in the way the poster claims them to be.

The images they have posted will not trigger the system.

If you want to debate the ethics of other aspects of what Apple is doing, there are plenty of threads elsewhere. This thread is about a false claim about a vulnerability in the system.

False claims about the vulnerabilities don’t help us to reason about what the risks actually are and detract from the moral or ethical debate.