[jl6]

For some reason, after reading the initial reporting on this system, I thought it was running against any photos on your iPhone, but now I read the actual paper, it seems like it only applies to photos destined to be uploaded to iCloud? So users can opt out by not using iCloud?

[foerbert]

Much of the discussion is about how trivial it would be for Apple to start scanning any photos on the phone at a later date.

Right now they are able to bill this as doing what they currently do server side, but client side. Later, they can say they are simply applying the same "protections" to all photos instead of merely the ones being uploaded to iCloud.

[nicce]

They can do it already. System is full black box, and all we have is their word. So, saying that adding something might enable something else, is not strong argument.

[foerbert]

By that logic we may as well never question anything any of these companies - or even governments really - do because they might just find a way to do it secretly and maybe nobody would ever figure it out.

[tandav]

Friendly reminder: until ios source code is closed all privacy claims is only backed by trust. They easily can do whatever they want if you're not compiling from source.

[cmsj]

This isn't really true in a world where it's trivial to reverse engineer and decompile binaries.

For example, we already now have a tool for generating NeuralHash hashes for arbitrary images, thanks to KhaosT:

https://github.com/khaost/nhcalc

[RegnisGnaw]

Also don’t upload to MS, Google, Dropbox as they also scan for CSAM.

[NTroy]

If Apple is to keep their word about guaranteeing the privacy of non-CSAM photos (which this whole discussion is about them not doing a very good job of), then they would only be able to do that with photos stored in iCloud because of this technical specification as to how the identification process works. That being said, other photos across your device are still monitored in a different way. For example Apple will scan photos that you send or receive via iMessage to automatically detect if they're nudes, and if you're underage, they will block them/send a notification to your parents.

[zimpenfish]

> Apple will scan photos that you send or receive via iMessage to automatically detect if they're nudes

Only if they're being sent to or from a minor, I thought?

[sharikone]

Did you ever experience that you turned some setting off but it was "accidentally" turned on again after some update/reboot?

[dathinab]

As far as I know apple plans to put up 2 systems, one focused on phones of people age < 13 which filters "more or less" any photos and uses AI to detect explicit photos and one which looks for known child pornographic photos and for now seems to not necessary apply to all photos.

But I haven't looked to closely into it.

[bengale]

Yeah this is basically it.

They have a system that checks for hashes of images to try and find specific CSAM from a database when images are uploaded to iCloud, this already happens but is now moving on device. When explaining this I've used the analogy that here they are looking for specific images of a cat, not all images that may contain a cat. When multiple images are detected (some threshold not defined) it triggers an internal check at apple of details about this hash and may then involve law enforcement.

The other one is for children 12 and under, that are inside a family group. The parents are able to set it up to show a pop up when it detects adult content. In this case they are looking for cats in any image, rather than specific cat image. The popup lets them know it may be an image not suitable for kids, that its not their fault and they can choose to ignore it. It also lets them know if they chose to open it anyway their parents will get notified and be able to see what they've seen.

This is a good rundown: https://www.apple.com/child-safety/

[avianlyric]

Yeah pretty much. Another way of thinking about it, is that to upload an image to iCloud, your phone must provide a cryptographic safety voucher to prove the image isn’t CSAM.