[f38zf5vdt]

Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary?

Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.

[d35007]

Apple said that the probability of a collision is quite a bit higher than that:

> As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment

The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review:

> Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field.

source: https://www.apple.com/child-safety/pdf/Security_Threat_Model..., page 10