Really disgusting idea: I wonder if it's possible for someone to use this as a 'discriminator' in a GAN to configure a generator to recreate the CP this is trying to avoid distributing in the first place.
That's not really correct. That's a forcibly created colliding image, that's not the output of the NeuralHash. Also as reported elsewhere, it's absolutely possible to do so.
It might have to do with the output possibly being a probability vector as opposed to a binary hash. The whole thing is thus differentiable and optimizable (if a dog image was incorrectly placed in the bad hash bucket it might only be on the border of it while the real CP corresponding to the hash is found at the probabilistic maxima of the hash bucket). Just guessing.
Where am I supposed to look in that pdf to understand that it isn't correct or credible? It is certainly true that the model has differentiable and thus optimizable outputs.
Are you trying to stop abuse of children, or enforce a standard that the idea of images of children is bad?
If you’re actually trying to stop abuse, having the computer create fake CP seems like an ideal outcome, since it would avoid the need for abuse of children.
Flooding the market with fakes and then directing consumers of the fakes to whatever mental health resources are available seems like it would fit the claimed problem far better than what apple is currently trying.
With the right algorithm you can turn any certain string of bits into a certain other string of bits. So is the image in the data, or is it really in the algorithm?
If the decoder was "trained" on and only works with predictable data, then it might be the algorithm that's illegal, but if a completely new illegal image is created, hashed, fed into the decoder and the decoder produces a valid illegal image, then the illegal data must be in the input, not the algorithm.
This is basically rule 1 of testing neural networks: if the testing data is different from the training data and the results are still correct, your network is "reading" the data correctly and not just memorising a list of known values. I guess this means you'd also need to prove that the decoder doesn't turn most hashes of non-illegal images into illegal images, but if you also did that, you'd have a pretty strong case that the illegal data is in the hash.
> if there's a possibility of e.g. faces on fbi's most wanted being snuck into the dataset
Sure, it's possible, but that doesn't seem to have happened in the past decade of PhotoDNA scanning cloud photos to match hashes provided by NCMEC - why would it suddenly start happening now?
You really don't understand the difference in scale distributed sensor netwise between the two different capabilities do you?
Server centric is the primitive that gives you periodic batch. Client resident let's you build up a real-time detection network.
Also, as they say in the financial world: past performance is not indicative of future results. No one would have thought to do so because this step hadn't been done. Now that this step has been done it is an easier to sell prospect. This is how the slippery slope works.
What's the realistic difference here between "my phone scans the photo on upload to iCloud Photos" and "iCloud Photos scans the photo when it's uploaded"?
Latency of upload doesn't come into play here because the scan results are part of the uploaded photo metadata; they're not submitted distinctly according to Apple's technical description.
(And given the threshold needed before you can decrypt any of the tagged photos with the client side system, the server side scanning would be much more "real-time" in this case, no?)
[0]: https://user-images.githubusercontent.com/1328/129860810-f41...
[1]: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...