[robertoandred]

The parent was proposing to “just send known CSAM”.

But OK, say someone sends you a sunset that fools the hasher. Then what? Of course one match won’t do anything, so you’d need to download however many matching sunsets. Then what? The Apple reviewer would see they’re sunsets and you’d challenge the flag saying they’re sunsets. And if somehow NCMEC got involved, they’d see they’re just sunsets. And if law enforcement got involved, they’d see they’re just sunsets.

These proofs of concept might seem interesting from a ML pov, but all they do is just highlight why Apple put so many layers of checking into this.

[heavyset_go]

> But OK, say someone sends you a sunset that fools the hasher. Then what? Of course one match won’t do anything, so you’d need to download however many matching sunsets. Then what?

A real attack would be to take legal porn images and make them collide with illegal images, so when a human goes to review the scaled down derivative images, those images very well look like they could be CSAM. Since there are many of them, they'd get sent to law enforcement. Then law enforcement would raid the victim's home and take all of their electronic devices in order to determine if they can be charged with a crime or not.

[beachy]

This where the "fog of war" kicks in. What with doors being busted down, police departments making press releases, etc. I can easily imagine that the victim could be prosecuted, convicted and sent away because no-one understood the subtlety that their legal porn was not in fact CSAM.

[simondotau]

The fog of war is largely in the realm of post-puberty minors, photos of which are not being included in Apple's corpus of hashes. I find it difficult to believe that anyone could mistake or otherwise "fog of war" a photograph of an adult and a prepubescent minor.

And that's assuming someone develops a hash collision which doesn't substantially mangle the photograph like the example offered on Github.

Specifically, only images categorised as "A1" are being included in the hash set on iOS. The category definitions are:

  A = prepubescent minor
  B = pubescent minor
  1 = sex act
  2 = "lascivious exhibition"

The categories are described in further detail (ugh) in this PDF, page 22: https://www.prosecutingattorneys.org/wp-content/uploads/Pres...

[Syonyk]

> Specifically, only images categorised as "A1" are being included in the hash set on iOS.

Do we know that for sure?

Apple has changed their mind enough times in the last week and a half that I'm convinced they're in full on defensive "wing it and say whatever will get people off our backs!" mode.

You can't read the threat modeling PDF and conclude that it was run through the normal Apple document review process. It reads nothing like a standard Apple document - it reads like a bunch of sleep deprived people were told to whip it up and publish it.

[simondotau]

That document is over six years old. It has nothing to do with Apple.

[beachy]

I don't really want to do the research, so I'll take your word for it.

But by fog of war I was thinking more like the victim already has some sleazy (though marginally legal) stuff on their computer, or a search led to a find of pot in their house, or they lied to try and get out of the rap, or perhaps the FBI offered them a deal and they took it because they saw no way out, or perhaps they were simply an unlikable individual who the jury took a dislike to.

Basically that things are not always clear cut, and they come out of the wrong side of things, in a situation created by Apple's surveillance.

[simondotau]

Even if I grant all of the above, I don't see how any of that is impacted by the distinction between on-cloud scanning and on-device scanning of photos which are being uploaded to the cloud.

Surveillance is surveillance. It's a bit more obnoxious that a CPU which I paid money for is being used to compute the hashes instead of some CPU in a server farm somewhere (which I indirectly paid for) but the outcome is the same. The risk of being SWAT-ed is the same.

[systemvoltage]

It would still be mentally draining to be accussed of CP. Can you imaging how terrified one would be if they see a warning message with a blurred sunset? I don't know exactly how the system works but from Apple's press release, it hides the image and gives a warning to the user. This would not go well on social media.

[kuratkull]

Remember, while you are refuting all this to each party, you are actually in the process of defending yourself against one of the worst criminal accusations possible. Your life will be investigated, your devices will be investigated - the amount of stress and reputational harm this causes is insane.

[netr0ute]

The point isn't to trick NCMEC, but rather create a DoS attack so no actual triggers can get through the noise.

[robertoandred]

I thought the point was to SWAT some innocent person? The goal keeps changing.

[riffraff]

But who would want that?

We all want privacy but it seems odd to try to DoS this, with high risk for yourself and very little to gain.

Might be useful when the system turns into mass political surveillance tho.

[mukesh610]

As I've commented elsewhere, DoS can be easily mitigated by implementing another layer with basic object recognition to filter out false positive collisions.