[simonh]

Of course it would be possible to implement content search, profiling and reporting mechanisms for such content, but this seems to be a singularly bad platform for that sort of search.

The image profiles are part of the OS so there's no mechanism to deliver image profiles separately for different countries. Also when the threshold number of matching images is reached, the matches are reported to a manual reviewer at Apple not a government. It only checks images on upload to iCloud photo storage.

So of course each of these limitations of the system could be changed, but you'd really need to change all of them and at that point you've created a completely different system. There's no simple change to this system that would suddenly turn it into a snitch for e.g. China or Saudi Arabia.

I've seen exactly the same objections raised every time any kind of device content search has become mainstream. Back in the 90s it was virus checking (Do you trust the AV company? What if they were bribed by the content companies?), full device indexing and search (Do you trust the OS vendor? What if they're in league with the government?). I'm very surprised this didn't blow up when Apple implemented ubiquitous image text recognition. Maybe it did. AV and device indexing mechanisms, which are ubiquitous, seem like a far more vulnerable target for such requirements.

So I don't really buy the slippery slope argument. In theory any government could pass a law requiring any company operating in it's jurisdiction to do anything, with an implementation suitable to that actual purpose. Of course this mechanism is motivated by laws in the US so it's a perfect example of exactly that, and it's a completely new system not a slippery slope subversion of an existing one. The real slippery slope here is legislative, not technical and I think that should be far, far more concerning.

I do think the legal and moral questions about this mechanism are legitimate. I think it would make more sense for Apple to scan photos in their cloud storage on the cloud storage rather than on upload. I understand there are theoretical privacy benefits to users from this implementation but the optics of having user's devices snitch on them are all wrong.

[luckyandroid]

>Back in the 90s it was virus checking (Do you trust the AV company? What if they were bribed by the content companies?), full device indexing and search (Do you trust the OS vendor? What if they're in league with the government?)

These are examples of companies choosing to do something as a selling point of their software as a benefit to the end user, and people worrying that it could aid the government down the line if they change their mind.

Apple's content review change is explicitly FOR reporting people to police in a way that can be expanded beyond it's currently set purpose (child porn) later.

>I'm very surprised this didn't blow up when Apple implemented ubiquitous image text recognition.

I'm personally not a fan of that stuff anyway, but personally if it's only my local device I don't tend to care about image recognition, it's only when it involves communicating information from MY hardware to THEIR servers that I get antsy.

[simonh]

>Apple's content review change is explicitly FOR reporting people to police in a way that can be expanded beyond it's currently set purpose (child porn) later.

I think it would be very hard to expand this beyond it's currently intended purpose, for the reasons I've given. It's terrible for identifying dissidents because it only catches them if they upload to iCloud servers. Dissidents are much more likely to be tech savvy than random child molesters. The reports have to go through Apple, and don't go directly to the cops. Also it's a global image profile list so it's not possible to keep country specific updates secret.

An effctive surveillance mechanism would need to change all of these.

[Nicksil]

>It's terrible for identifying dissidents because it only catches them if they upload to iCloud servers.

This is a configuration change. Without knowing the implementation, I'd bet a lunch that, for the time-being, the reason this thing is executed only upon upload to iCloud is because there's some simple business logic buried in there telling it to do so.

>Dissidents are much more likely to be tech savvy than random child molesters.

This is a curious argument. You didn't explain why you think this might be. What is it about a dissident that makes him or her more savvy than some random child molester?

>An effctive surveillance mechanism would need to change all of these.

If true, the obstacles you outlined are trivial to overcome.

[simonh]

>This is a configuration change.

Not it isn't, the check is built into the upload client, they'd have to implement an on-device storage scanning mechanism. That's a different type of system implemented in a different kind of service.

Not that doing that is hard at all, it's not rocket science and they already have full-system indexing and search, but that's also why this isn't a significant step down any kind of technical slippery slope. The problem here is legislative, not technical.

[aNoob7000]

Apple should just scan the pictures that are in iCloud (their servers). They just assumed that if you have the iCloud option enabled on your device that it gave them the right to do the scan on your phone/computer.

I want to also point out that A/V companies never said they were going to scan for child abuse images on your computer and report you if they found any.

Like you said, the optics are terrible.

[nicce]

> Apple should just scan the pictures that are in iCloud (their servers). They just assumed that if you have the iCloud option enabled on your device that it gave them the right to do the scan on your phone/computer.

End result is the same. Difference is, that now Apple has very limited access to your images. You can only trust in closed systems. When you step into the Apple ecosystem, you are giving a lot of trust.

> I want to also point out that A/V companies never said they were going to scan for child abuse images on your computer and report you if they found any.

Why would they say, if it is perfectly legal to do anyway. They literally scan every file, so no need to mention anything specific which could lead only for negative PR.

[zionic]

Apple has unlimited access now, all that stands in their way is the thinnest of policy lines.

[nicce]

> Apple has unlimited access now

Always been. You don't own your iOS based device which is very closed source and mostly unusable for any other operating systems.

[LatteLazy]

>The image profiles are part of the OS so there's no mechanism to deliver image profiles separately for different countries

Haven't Apple already said it WILL be country specific?

>Apple’s new feature for detection of Child Sexual Abuse Material (CSAM) content in iCloud Photos will launch first in the United States, as 9to5Mac reported yesterday. Apple confirmed today, however, that any expansion outside of the United States will occur on a country-by-country basis depending on local laws and regulations.

https://9to5mac.com/2021/08/06/apple-says-any-expansion-of-c...

I think they'd need to be country-aware at least, otherwise the FBI or whoever will get reports for all people on earth when they presumably don't need them for anyone outside the US?

[simonh]

Reporting is country specific and US only yes, but the profiles are delivered baked into the OS. I suspect this is so that pedophiles can't buy a phone mail order from Canada and bypass the system.

[LatteLazy]

I think the profiles will need to be country specific too. What counts as CSAM in some places doesnt in others (here in the UK we have a ban on cartoons but bath pics are allowed for instance).

This is something Apple have been pressed on a lot. So far (I'd be happy to be corrected) they've only said "whatever local law permits". That sounds ok, till you realise Saudi will want gays reported and China wont like any Winnie the Pooh pics...

[simonh]

China already operates their own iCud storage so this is irrelevant to them.

Apple doesn't have any iCloud data centres in Saudi, so Saudi can't pass laws about what is or isn't stored in them.

Look, the way this works and how it's implemented matters. It's stunning to me how many people are thoroughly confused and jump to unwarranted conclusions about how this actually works and what that means.

[LatteLazy]

I dont think your saudi or china points grasp the nature of this tech. This is about checking what users have on their devices BEFORE it is uploaded to icloud.

So both China and the Saudis (any plenty of other governments) will be very interested as right now, it takes a lot more effort for them to access phone contents (there certainly aren't mass surveillance programs like this for handsets).

I weirdly agree with your last paragraph, but i think we disagree about the details. I can't find any evidence for your assessment that this can only be used against 1 (US) set of image hashes. Or that shitty regimes won't be allowed to abuse it.

If Apple came out and proved that, i might not be happy but my worst fears would be gone. Their silence is sort of deafening at this point...