[ReactiveJelly]

"Now you can block people in Drive. To prevent people from sharing unwanted files with you, ..."

hahaha. What a coincidence, Google!

So you got a hold of the neural hashes, and then used an error function and descent to generate images that match a 'hash'?

It feels wrong to call them 'hashes' when they're so weak to pre-image attacks. They're not the same idea as cryptographic hashes at all.

Also want to underline how spooky it is that some of them do resemble human forms.

[411111111111111]

Some? Literally all are clearly based on pornography.

First is veg&butthole, then boobs, next is doggy style etc etc (edit: it seems the order isn't consistent. So I'm likely seeing different images then you.)

You can go through them all and see the original pornography if you look at the shapes. To me, it looks more like they started with the real images and tweaked them to make them artsy.

[yesbabyyes]

I don't know, this sounds a bit like a Rorschach test to me.

[ASalazarMX]

I love this comment. I have no idea how these were generated, but even starting with random noise it's possible to end up with vaguely human shapes if that's what originated the hashes to begin with.

These images could be a joke, as I don't think we have a clear technical documentation of how these hashes are generated. Computer vision? Vectors? Face recognition software? It's definitely not a naive hash.

Edit: seeing the other comments in this thread referencing Twitter, it looks like it's more naive than expected, as the hash is resistant to resizing, but not to cropping. The implementation can change at Apple's discretion, though.

[dragonwriter]

> They're not the same idea as cryptographic hashes at all.

There is a reason cryptographic hashes are distinguished; some applications of hashing are only concerned with minimizing non-malicious collisions.

(Arguably, this is an application where malicious collisions are an issue, but perceptual hashes don't purport to be cryptographic.)

[baybal2]

Don't forget how it started at Google.

Long before they claimed that they scan email for child porn, they made an email scanner to appease China, on a condition they will not target dissidents.

I think all remember how it went. Seeing their intimidation work, it only fired up the Chinese government, and led them to only increase their attempts at arm twisting, until Google clumsily pretended to "be tough" while still doing their last attempt behind the scenes negotiations, which, to their big surprise, got them banned overnight.

[panarky]

Don't forget how Google admitted their mistake and withdrew from the Chinese market.

Even though Google could have made a ton more money by helping China to build the tools of repression.

Don't forget that other large US corporations like Microsoft, Apple and Activision do build censorship tools and participate in repressing dissent.

[woofie11]

My recollection was that Google was outgunned by Baidu and similar Chinese tools, who were being actively supported by the CCP, while Google was half-tolerate. At the same time, it was losing security, IP, and reputation. There was a nice PR play around it, but I don't think "Google could have made a ton more money by helping China to build the tools of repression." It's better to build those in-house.

[baybal2]

> Don't forget how Google admitted their mistake and withdrew from the Chinese market.

Which I point to the decision they came inadvertently. It was their intention to play games with the regime which backfired on them, not vice versa.

[Applejinx]

I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin.

Not sure exactly how you'd go about doing it, but it seems like there might be a process for 'evening out' areas into solid color that maintains the hash? In which case you're running extensive image processing on illegal images and making variations from those very images.

More info on how this is done?

[dragonwriter]

> I assumed that these are reverse engineered from legitimately illegal and problematic porn of known origin

I would assume these were engineered by getting the perceptual hash valies, using distance from the hash values in the DB as an error function, and starting with an innocuous image and hash value, and iterating to a collision for each.

[zepto]

That technique can’t fool Apple’s algorithm.

For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point.

[dragonwriter]

> For what it’s worth, the null hypothesis is that they are just fakes and the commenter is at best trying to illustrate a point.

No, that's not “the null hypothesis”. It is a positive claim.

[zepto]

Yes it is the null hypothesis.

The poster is making a positive claim without evidence. Indeed the claim is unverifiable.

Reasonable priors lead fo a null hypothesis that they are at least simply mistaken.

This is without even taking into account other indicators of credibility or authority, or perverse incentives, as priors.

This is a rational use of ‘null hypothesis’, but it also matches the scientific use, which would be that the claim is spurious unless experiment shows otherwise.

In any case, we know that the poster is in fact wrong in their claim.

[dragonwriter]

> Yes it is the null hypothesis.

No, its not.

> The poster is making a positive claim without evidence.

True.

That doesn't make the alternate positive claim you have posited into “the null hypothesis”.

A null hypothesis is null. What you are stating may be your prior, but it is not the, or even a valid, null hypothesis.

> This is a rational use of ‘null hypothesis’, but it also matches the scientific use,

“Null hypothesis” is a very specific scientific term of artz it has no other meaning.

And, no, the specific counternarrative presented here does not match the scientific use of “null hypothesis”.

[salawat]

How do you know? Do you have access to Apple's algorithms and an account to generate enough hits, and access to the safety vouchers and decryption system to verify your assertions?

I mean, if you're calling someone out, at least provide some evidence yourself. Short of a reproducible outcome, you're just as questionable in conclusion as the poster.

[zepto]

No. I have access to the published information on how the system works, and I have access to the poster’s claim.

The poster’s claim is false based on what they have said.

> you're just as questionable in conclusion as the poster.

Not correct. You don’t need evidence to disprove a claim that is logically false. The poster’s claim is logically false.

Here is a copy of the explanation I gave elsewhere:

—-

I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.

What prevents a generated image from matching both is that the attacker would need to know what the image they are trying to spoof looks like, in order to make a false positive of both. I.e. the attacker would need a copy of the original CSAM, and the spoofed file would end up looking like it could be at least plausibly mistaken for that exact image.

[ac29]

> I can be certain because I have looked at the images, and they are obviously not CSAM. Since the visual derivative is generated from CSAM, any spoof must look like it could be mistaken at a glance for CSAM.

Isnt this making the relatively huge assumption that humans and Apple's algorithms have the exact some opinion of what something "looks like"?

[Applejinx]

Followup, since I can't edit: if my assumption isn't correct, well then, I stand corrected. I said in past tense, 'I assumed', and then asked for more info. That's not forthcoming, just a bunch of very upset assertions that of course I'm wrong and these things can't be reverse engineered from real porn.

I'm sure not interested in proving they can. Mind furnishing the info about how it's really done, then? Since according to you (for very obvious reasons) you can never compare these images to the source for the hashes, where did you extract the hashes from?

If you can so easily reverse engineer false positives from random data without ever seeing or using genuine porn to produce it, shouldn't you be disseminating this content as widely as you possibly can, rather than warning people about the danger of interacting with these false-positive images?

Still puzzled how and why this is being done. Are you trying to render Apple's system useless, or not?

Summary: I'm saying "there may be a way to take existing images that are illegal even to possess, and process them to obliterate the image while maintaining the hash. Is that what's being done here?" and the response is "AM NOT!!"

[YokoSix]

Genuine question: If those image were really generated from illegal porn, are those images themselves considered illegal? Or in other words: How much do you have to modify illegal images for them to become legal again? Or do they stay illegal no matter how much you transform them?

[f38zf5vdt]

Looking at the script below, it looks like it uses a gradient function for loss so that it learns to approach an image that generates a collision. If the case that the hashes themselves, being a result of a neural network, can be reverse engineered into pornographic images then does that raise a legal quandary?

Apple said that the risk of collision is "1 in one trillion" which for a hash function would be terrible. We also don't know what the one trillion images they tested against were. If you upload your regular porn to iCloud, it's likely that pornographic images will raise more false positives than say, pictures of sunsets.

[d35007]

Apple said that the probability of a collision is quite a bit higher than that:

> As the system is initially deployed, we do not assume the 3 in 100M image-level false positive rate we mea- sured in our empirical assessment

The "1 in 1 trillion" part is the probability that the number of false positives could exceed the threshold needed to trigger a human review:

> Apple always chooses the match threshold such that the possibility of any given account being flagged incorrectly is lower than one in one trillion, under a very conservative assumption of the NeuralHash false positive rate in the field.

source: https://www.apple.com/child-safety/pdf/Security_Threat_Model..., page 10

[Applejinx]

Also relevant question: if these images were not at all generated from illegal porn, but they connect to hashes being used to flag illegal porn, is the purpose of this exercise to generate methods to SWAT people over the internet?

As in, pursue a mechanism to get these onto somebody's computer in a way that they'll be backed up via iCloud (for instance, if a person's got their email account including trash folder backed up in iCloud, and you send them the pictures which they 'throw away' because it means nothing to them, placing the images in a trash folder in the mail preferences)

Is that (a) practical and (b) the intent of this exercise? Seeing as every question I've had here has led to karma burning I figured I'd double down and ask if the person doing this is trying to prepare a weapon for swatting people. There are times I respond to downvoting pressure to 'stop talking!' by getting more interested, which I'm sure is a common reaction among some hackers.