Some? Literally all are clearly based on pornography.
First is veg&butthole, then boobs, next is doggy style etc etc (edit: it seems the order isn't consistent. So I'm likely seeing different images then you.)
You can go through them all and see the original pornography if you look at the shapes. To me, it looks more like they started with the real images and tweaked them to make them artsy.
I love this comment. I have no idea how these were generated, but even starting with random noise it's possible to end up with vaguely human shapes if that's what originated the hashes to begin with.
These images could be a joke, as I don't think we have a clear technical documentation of how these hashes are generated. Computer vision? Vectors? Face recognition software? It's definitely not a naive hash.
Edit: seeing the other comments in this thread referencing Twitter, it looks like it's more naive than expected, as the hash is resistant to resizing, but not to cropping. The implementation can change at Apple's discretion, though.
Long before they claimed that they scan email for child porn, they made an email scanner to appease China, on a condition they will not target dissidents.
I think all remember how it went. Seeing their intimidation work, it only fired up the Chinese government, and led them to only increase their attempts at arm twisting, until Google clumsily pretended to "be tough" while still doing their last attempt behind the scenes negotiations, which, to their big surprise, got them banned overnight.
My recollection was that Google was outgunned by Baidu and similar Chinese tools, who were being actively supported by the CCP, while Google was half-tolerate. At the same time, it was losing security, IP, and reputation. There was a nice PR play around it, but I don't think "Google could have made a ton more money by helping China to build the tools of repression." It's better to build those in-house.
I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin.
Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images.
> I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin
I would assume these were engineered by getting the perceptual hash valies, using distance from the hash values in the DB as an error function, and starting with an innocuous image and hash value, and iterating to a collision for each.
The poster is making a positive claim without evidence. Indeed the claim is unverifiable.
Reasonable priors lead fo a null hypothesis that they are at least simply mistaken.
This is without even taking into account other indicators of credibility or authority, or perverse incentives, as priors.
This is a rational use of ‘null hypothesis’, but it also matches the scientific use, which would be that the claim is spurious unless experiment shows otherwise.
In any case, we know that the poster is in fact wrong in their claim.
How do you know? Do you have access to Apple's algorithms and an account to generate enough hits, and access to the safety vouchers and decryption system to verify your assertions?
I mean, if you're calling someone out, at least provide some evidence yourself. Short of a reproducible outcome, you're just as questionable in conclusion as the poster.
No. I have access to the published information on how the system works, and I have access to the poster’s claim.
The poster’s claim is false based on what they have said.
> you're just as questionable in conclusion as the poster.
Not correct. You don’t need evidence to disprove a claim that is logically false. The poster’s claim is logically false.
Here is a copy of the explanation I gave elsewhere:
—-
I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.
What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like, in order to make a false positive of both. I.e. the attacker would need a copy of the original CSAM, and the spoofed file would end up looking like it could be at least plausibly mistaken for that exact image.
Followup, since I can't edit: if my assumption isn't correct, well then, I stand corrected. I said in past tense, 'I assumed', and then asked for more info. That's not forthcoming, just a bunch of very upset assertions that of course I'm wrong and these things can't be reverse engineered from real porn.
I'm sure not interested in proving they can. Mind furnishing the info about how it's really done, then? Since according to you (for very obvious reasons) you can never compare these images to the source for the hashes, where did you extract the hashes from?
If you can so easily reverse engineer false positives from random data without ever seeing or using genuine porn to produce it, shouldn't you be disseminating this content as widely as you possibly can, rather than warning people about the danger of interacting with these false-positive images?
Still puzzled how and why this is being done. Are you trying to render Apple's system useless, or not?
Summary: I'm saying "there may be a way to take existing images that are illegal even to possess, and process them to obliterate the image while maintaining the hash. Is that what's being done here?" and the response is "AM NOT!!"
Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?
Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary?
Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.
Apple said that the probability of a collision is quite a bit higher than that:
> As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment
The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review:
> Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field.
Also relevant question: if these images were not at all generated from illegal porn, but they connect to hashes being used to flag illegal porn, is the purpose of this exercise to generate methods to SWAT people over the internet?
As in, pursue a mechanism to get these onto somebody's computer in a way that they'll be backed up via iCloud (for instance, if a person's got their email account including trash folder backed up in iCloud, and you send them the pictures which they 'throw away' because it means nothing to them, placing the images in a trash folder in the mail preferences)
Is that (a) practical and (b) the intent of this exercise? Seeing as every question I've had here has led to karma burning I figured I'd double down and ask if the person doing this is trying to prepare a weapon for swatting people. There are times I respond to downvoting pressure to 'stop talking!' by getting more interested, which I'm sure is a common reaction among some hackers.
These harmless generated images have a neuralhash equivalent to those provided in the NCMEC database submitted for testing.
I repeat: Dont upload these harmless images to iCloud as Apple will assume its Child Porn (CSAM).
Scripts were available on a GitHub repo but were removed because they may cause damage to others.
Based on the documentation from Apple, they are waiting to get *several* matches, *not only one* (we don't know what is *several* but I don't expect something like <= 3 pictures).
Once the rate has been reached, they ask to a physical team to review the "positive matches", and deliberate if, yes or no, the images are CSAM or not.
If yes, after the manual process, the authorities are called.
The database of 200_000 images used by Apple (and others?) is private, and I did not found any trace of the hashes (but I could made a mistake here).
So, how do you know that those correspond exactly (or with a certain threshold that has NOT been disclaimed by Apple) to the CSAM DB?
They way I see it, this is the best photo backup approach one can possibly take. Just get flagged for child porn, and have all your iPhone photos stored indefinitely on FBI servers.
The problem with Apple’s approach to CSAM is that they use Neuralhash. Unlike other simple perceptual hashes, the failure modes and the collisions using this method are not well understood. I repeat here my previous comment in another thread : they use NN and triplet embedding loss, the exact same techniques used by neural networks for face recognition, so maybe the same shortcomings would apply here. For example a team of researchers found a 'Master Faces' that can bypass over 40% of Facial ID. Now suppose that you have such an image in your photo library, it would generate a ton of false positives and not just a single match with the NCMEC database.
You have misunderstood. NeuralHash is the visual derivative. Read [1] carefully, it's a very confusing document even for experts - nowhere is there a second step to this process where some second type of "visual derivative" is matched.
Is this true? I have no idea how to even test it without causing problems to myself. After all, they pushed me to give them my credit card and physical address.
This is so fake it's not even funny. These are just images generated by the model from https://thisartworkdoesnotexist.com . It's hilarious to see so many people falling for it here
I've never used modern apple products, but I have a question on how apple cloud works: is it possible that simply sending these in a messenger software to someone who uses apple cloud for automatic backups could get that person flagged as a child pornographist?
The thing that everyone has their panties in a bunch about here and a that like an antivirus scanner, there will be a hash match to child abuse images when you send it.
The current practice is that Apple, Google, Microsoft, etc scan the content of your cloud storage.
The scenario that you described is a risk and has been since cloud providers started scanning 10-15 years ago. Some large companies scan their file servers as well.
I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.
What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like, in order to make a false positive of both. I.e. the attacker would need a copy of the original CSAM, and the spoofed file would end up looking like it could be at least plausibly mistaken for that exact image.
There are only so many ways features can be permuted. The I'll defined nature of NN's requires the manuals step because of a neural hash collision.
My challenge to you is this: what stops this system from being abused for non child pornography purposes?
The answer is: nothing.
That's what has people's knockers in a twist. It is a backdoor, invisibly crafted, waiting to be subverted by an abusive power that manages to get into an advantageous enough position.
Arguing that Apple's algorithms are fine misses the point. The behavior should not exist.
No, messages and regular file backups aren’t checked for child porn and the online photo library has multiple checks and reviews to prevent any false flagging.
then millions of people can f** * because it's apple! didn't google recently close a personal account of an indie game developer without giving him any explanation?
Yeah, seems like anyone clicking through is playing with fire. If the description is correct, posting the link is highly irresponsible. (It seems like the right thing to do would be to serve the content from a server the OP controls themselves.)
Maybe I'm an idiot but my curiosity got the best of me and I clicked the link. The photos just look like abstract modern art, although their perceptual hash may match that of known CSAM, I doubt that anyone who clicked the link will get into legal trouble even if Google flags that folder by detecting a perceptual hash match, as they will likely use real people to verify before taking legal action.
Google has been recently focused on cultivating the image that they care about user privacy. The last thing they want to do is call the cops on a bunch of HN users for looking at some abstract swirly pics.
I think the concern here isn’t legal trouble but getting banned by some automated system.
I’ve heard from googlers that once an account is nuked for suspected child abuse no one will ever want to touch it to find out whether the ban was legitimate.
This post is still getting linked from other places, so I think it's helpful to point out that it's almost certainly fake. (Hence its [flagged] status)
The images shown do appear to be adversarially generated inputs against some NN-based image hash or classifier, but there is no evidence to suggest that this is at all related to Apple's NeuralHash, or that the colliding hashes are from a real CSAM database (the target hashes are not public).
OP claimed they would "release 5 pieces of proof in the next 5 days" [1], and guess what, 11 days later they still haven't.
Look at OP's post and comment history, it's quite clear that they are a troll.
In the mean time, it has been actually proven that hash collisions against NeuralHash are trivially possible, see [2]
Why was this flagged? I'm sure there are good reasons, I just want to know.
EDIT: Oh, I mixed up tabs. This is a link to a google drive of pictures. Because I have scripts disabled, I got no thumbnails, and I'm thinking since this was flagged, maybe I really don't want to get any thumbnails.
/r/Apple talks about this topic a lot and, similar to HN, is not happy about it. This drive link brings very little additional light to what was already known and discussed.
The sub is in crisis mode right now. Normally a huge pro-apple even on their worst days do no wrong sub. Right now people are pissed and speaking up. The mods are pissed that they have to deal with it (boo hoo). Anything to add fuel to this fire is good in my eyes because the same people who said "It's ok I have nothing to hide" from Edward Snowdens work are getting slapped in the face for their same ideology to this happening to Apple.
I guess I meant that figuratively. The same genre of people who thought prism is ok because they have nothing to hide to give up privacy are now the same genre that think this privacy invasion is ok as well.
What hash algorithm are they using? If this is legitimate (I’ll be honest - I’m not clicking) then surely any hash this easy to pre-image attack is completely useless? Why wouldn’t they be using a cryptographic hash here?
As I understand it the NN "perceptual hash" is supposed to hash the image, not the file. eg, if I take a photo of my cat, and hash the file. Then remove my geo data from the exif - the hash no longer matches. It is still very clearly my cat, but cryptographic hashes don't match. This could be resizing the image, saving as png, mirroring/flipping it, etc.
The "perceptual hash" should be able to say "no, that's still the same image" while the file data has been entirely transformed.
Is this obtained from a set of images leaked from NCMEC and regenerated to match the expectations of the as yet (and probably forever) unknown Apple’s NeuralHash algorithm as well as the threshold used to flag content for internal human review for a system that’s going to be operational for U.S. Apple device users only when iOS 15 is released?
On what basis is a set of forest-like and post-alien-invasion and post-apocalyptic abstract art is going to get flagged (my poor eyes see one or two that could have some symbolism)?
As far as I know they all do CSAM, Google, Microsoft, Facebook, and now Apple with iOS 15. So isn't it already a problem that you have it in a Google Drive?
I think someone needs to play big techs own game and find some cases where this algorithm underperforms based on race or gender, publish a bunch of clickbait articles and get the whole program canceled. Erosion of privacy and authoritarianism isn’t enough to gain traction.
hahaha. What a coincidence, Google!
So you got a hold of the neural hashes, and then used an error function and descent to generate images that match a 'hash'?
It feels wrong to call them 'hashes' when they're so weak to pre-image attacks. They're not the same idea as cryptographic hashes at all.
Also want to underline how spooky it is that some of them do resemble human forms.