[Wowfunhappy]

They're searching for different things though. To my knowledge, before now iOS has never scanned for fingerprints of specific photographs. It would be so darn easy to replace the CSAM database with fingerprints of known tiananmen square photos...

[shuckles]

That is a distinction without a difference. I’m sure you could put together quite a good tank man classifier (proof: Google Reverse Image Search works quite well), and it’d catch variations which a perceptual hash wouldn’t.

The only difference is intent. The technical risk has not changed at all.

[dpedu]

That is to say face scanning is equally insidious as the new feature?

[shuckles]

The technical risk to user privacy - if your threat model is a coerced Apple building surveillance features for nation state actors - is exactly the same between CSAM detection and Photos intelligence which sync results through iCloud. In fact, the latter is more generalizable, has no threshold protections, and so is likely worse.

[int_19h]

It's the legal risk that is the biggest problem here. Now that every politician out there knows that this can be done for child porn, there'll be plenty demanding the same for other stuff. And this puts Apple in a rather difficult position, since, with every such demand, they have to either accede, or explain why it's not "important enough" - which then is easily weaponized to bash them.

And not just Apple. Once technical feasibility is proven, I can easily see governments mandating this scheme for all devices sold. At that point, it can get even more ugly, since e.g. custom ROMs and such could be seen as a loophole, and cracked down upon.

[shuckles]

This hypothetical lacks an explanation for why every politician has not demanded Apple (or say Google) do this scope creep already for photos stored in the cloud where the technical feasibility and legal precedent has already been established by existing CSAM scanning solutions deployed at scale.

[int_19h]

I have to note that one of those solutions deployed at scale is Google's. But the big difference is that when those were originally rolled out, they didn't make quite that big of a splash, especially outside of tech circles.

I will also note that, while it may be a hypothetical in this particular instance as yet, EU already went from passing a law that allows companies to do something similar voluntarily (previously, they'd be running afoul of privacy regulations), to a proposed bill making it mandatory - in less than a year's time. I don't see why US would be any different in that regard.

[mlindner]

Nonsense. Building an entire system as opposed to adding a single image to a database is a substantially different level of effort. In the US at least this was used successfully as a defense. The US cannot coerce companies build new things on their behalf because it would effectively create "forced speech" which is forbidden by the US Constitution. However they can be coerced if there is minimal effort like adding a single hash to a database.

[shuckles]

Photos intelligence already exists, and if people are really going to cite the legal arguments in Apple vs. FBI, then it’s important to remember the “forced speech” Apple argued it could not be compelled to make was changing a rate limit constant on passcode retries.

[zepto]

Exactly this. The whole thing is a red herring. If Apple wanted to go evil, they can easily do so, and this very complex CSAM mechanism is the last thing that will help them.

[shuckles]

I’ve read your comments, and they are a glass of cold water in the hell of this discourse. This announcement should force people to think about how they are governed - to the extent they can influence it - and double down on Free Software alternatives to the vendor locked reality we live in.

Instead, a forum of presumably technically savvy people are reduced to hysterics over implausible futures and a letter to ask Apple to roll back a change that is barely different from, and arguably better than, the status quo.

[zepto]

Thanks. I couldn’t agree more.

We need both - develop free software alternatives (which means to stop pretending the alternatives are good enough), and to get real about supporting legal and governance principles that would protect against abuses.

If people want to do something about this, these are the only protections.

[samstave]

So you have User A - they upload a pic with User A and Peoples B,C,D,E,Z

icloud scans for those faces

finds those faces and ties them to other ID accounts via face - then via fingerprint recognition to a device, and to a location based on IMEI etc.

Apple's platform is literally the foundation for the most dystopian digital tool-set in history...

Once the government is able to crack the apple war chest, everything is fucked.