[siscia]

So I was ignorant on the issue and completely against the approach of Apple.

Then HN taught me that any company storing images on their infrastructure in the US must report pedophilic images to the US government.

At this point, the approach taken by Apple seems like the best one to me, if you don't want to store pictures in clear on your servers.

What other technical approach are people advocating for?

Another point it is to try to change the law, but this is beyond the scope of the conversation.

[viktorcode]

The problem I have with this approach is that it introduces on-device scan for images. All what is needed to adopt it to scan for different kind of images is to connect it to different database, say, Winnie the Pooh memes featuring CCP chairman, and boom, jailed dissenters. And ability to scan all images is but a minor firmware update away.

Server scanning makes it clear that the company running the servers has access to your photos. So you can either find a form of encrypted storage, or be okay with that, depending on your privacy stance. Having device with ability to scan your photos removes that choice. It is a privacy invasion.

[hansel_der]

> ability to scan all images is but a minor firmware update away

ios already does on-device ml-based photo categorisation for some time, afaik no way to turn it off.

[romwell]

And now it's pretty much the same thing, but with a SWAT team knocking your door out when the ML messes up.

Yay progress.

[sumeno]

The SWAT team is knocking on your door after you've uploaded multiple instances of child porn to iCloud and those instances have been verified to actually be child porn by a human. That sounds fine to me.

[romwell]

>The SWAT team is knocking on your door after you've uploaded multiple instances of child porn

...or whatever gets sneaked into a database that nobody can take a look at, and whose maintainers have zero obligations to you.

>and those instances have been verified to actually be child porn by a human.

Yeah, SWAT teams doing their homework before shooting people up is precisely why SWATting is a completely innocent thing to do and never put anyone in danger.

And that also does nothing in case of "neural" (aka blackbox) hash collision, where the Algorithm mistakes a normal picture for CP. The "human" you have in your dreams doesn't have access to the actual file you have on your device, right? (At least, that's the sales pitch for on-device privacy). They won't know until they get you.

Personally, I would hope that HN people know better than to blindly trust an opaque algorithm running off an opaque database to never make a mistake in where it sends SWAT teams.. but here we are.

[sumeno]

The algorithm doesn't report to the SWAT team. It reports to Apple who verifies it.

[EveYoung]

But Apple only plans to scan photos that are synced with iCloud, don't they? So you could just switch to an E2E encrypted alternative and drop iCloud completely.

[unionpivo]

Yes but it probably took additional code to only scan pictures that are synced to iCloud.

Probably not monumental task, to change to scan every picture.

[nicce]

I have to remind again, that iOS is a blackbox, closed source system. All this speculation applies also for a moment before they added anything. They might have had this code ready for years already. All we have is what they say. It is already very trivial to scan everything on on your device and send that metadata. Few lines of code. At the moment when they say about scanning everything in phone publicly without opt-out, then we should be worried. Once again, there is no way telling what they are doing already.

[unionpivo]

The difference is now every government knows too.

They can't pretend they don't have the capability.

And if they can scan for CP, why can't they scan for "whatever" else instead.

[m4rtink]

This is not the first time they have run into this - due the to AppStore being a walled garden they are the sole gate keeper who decides what goes in and what not. Makes sure the users are safe and everything. Perfect, right ?

Well, until protesters want to use an app in the store to coordinate their protests yet the government wants you to reject it, so the protesters can't use it:

https://www.applefritter.com/content/teargas-walled-garden-i...

With users not being able to install the app themselves Apple is the single point of failure with no plausible deniability like Android (any any sane OS in general) has. And they did reject the app.

And just a few months before this happened I attended a talk about free software from FSF and they mentioned just the same thing about iOS and the gate keeper being the single point of failure a repressive regime can apply pressure on. Turned on to not be far fetched at all...

[avianlyric]

iOS has been running complex Neural Nets on all your images for years now. It powers all their social features and search.

Apple have always had the capability, and have been advertising it as central selling point of new versions of iOS for years. That ship sailed along time ago.

[croutonwagon]

Well.....

I think its more than that. images sent with iMessage are stored in iCloud, even if the device is not necessarily uploading.

How else would that have such warnings they claim in their announcement. [1]

And we have seen these systems have their scope/use case changed in the past [2]

To the point in the other discussion [3]. OP stated that Apples plans to scan and then upload suspected images are illegal. But i would think that they are only scanning images, client side, that users themselves are attempting to upload (either though attachments, or automatic iCloud backups etc) which would put Apple in the clear. In this case that would be iCloud images, or those that piggyback iCloud services like iMessage etc.

[1] https://www.apple.com/child-safety/ [2] https://www.eff.org/deeplinks/2020/08/one-database-rule-them... [3] https://news.ycombinator.com/item?id=28110159

[jeromegv]

Stop repeating this lie. iMessages photos are not part of this. This is written in the technical document. This is only photos from iCloud photos. It's been debunked, just read this article: https://daringfireball.net/2021/08/apple_child_safety_initia...

And of course the scope could change tomorrow. Just like the scope of Android could change tomorrow. They could even have changed the scope without doing an announcement!

[croutonwagon]

So there is really no need to be this aggressive.

In my comment history it clearly shows that there's an effort to parse through the information and seek clarity.

And its worth noting that iMessage data is and can be backed up to iCloud, and not just using backups. For many with multiple devices this is specifically useful.

https://support.apple.com/en-us/HT208532

Further, as to this

>And of course the scope could change tomorrow. Just like the scope of Android could change tomorrow. They could even have changed the scope without doing an announcement!

I am pointing out that there is a specific history of this already on record and documented. And their technical documents specifically state their intentions.

Page 3 : https://www.apple.com/child-safety/pdf/Expanded_Protections_...

"This program is ambitious, and protecting children is an important responsibility. Our efforts will evolve and expand over time"

I don't understand why you find such an observation so offensive. Its pretty clear Apple sees this as a first step into what will eventually be a much larger program.

[carom]

>The problem I have with this approach is that it introduces on-device scan for images.

Windows already does this via Windows Defender. This is a basic AV functionality and much more privacy preserving.

[DangerousPie]

But Windows Defender doesn't report you to law enforcement when it believes it found a virus.

[nicce]

How do you know that? It is blackbox paradox, and all we have is what they say. They might report CSAM hashes to law enforcements. Any file can be a threat, hence images are included for scans. Defender also uploads whole files as unencrypted if you don’t opt-out.

[vmladenov]

Neither does this.

https://www.howtogeek.com/719825/how-to-stop-windows-10s-ant...

If Microsoft receives an illegal file through this channel, they are legally obligated to report it in the US.

[thw0rted]

...if a human actually gets the file, figures out what type it is, and examines it for themselves, they'd be obligated to report it. With the number of Win10 devices in the world, how big would their security team have to be to hand-groom every automatically submitted "suspicious" sample? (For that matter, why would a vanilla JPG get flagged as "suspicious" in the first place?)

[avianlyric]

> All what is needed to adopt it to scan for different kind of images is to connect it to different database, say, Winnie the Pooh memes featuring CCP chairman, and boom, jailed dissenters.

The CCP have already throughly demonstrated that they don’t need manufactures consent to build these systems.

Look at the Uyghur population in China. They already have their phones scanned on device for dissident material, not by coercing manufacturers, but by forcing the population to install a surveillance app. Then making it illegal to use a phone without it.

Being caught at checkpoint without the app installed and working is grounds for immediate arrest and re-education.

[adventured]

> The CCP have already throughly demonstrated that they don’t need manufactures consent to build these systems.

It was obviously merely an example for illustration purposes by the parent. To get a point across it's often very helpful to use a stark, clear example.

Few governments will ever have the extraordinary capabilities and resources of the CCP in China.

For the other ~190 governments that will never reach that level of capability, what they might have now is a globe-spanning billion-device corporation like Apple more willing to assist them.

[542458]

Do mandatory reporter laws work like that? I was under the impression that you had to report something if you saw it, but you had no obligation to be actively scanning or to compromise encryption to do so. For example, I don’t think S3 does any active scanning and you can definitely shove any encrypted blob you want onto their servers with no obligation to give them a decryption key.

IMO this appears to be Apple either a) trying to preempt future criticism or regulation or b) responding to some behind-closed-doors pressure/bargaining with US authorities.

[nicce]

I think you have to be aware of what is happening, before you can say that nothing criminal happens. This is where scanning steps in. You can’t turn blind eye.

[amanaplanacanal]

There is a big jump from reporting criminal activity if you happen to see it, to actively searching it out. It is the jump from police arresting you if they see you smoking a joint to police searching your rooms to make sure you don’t have any cannabis in there.

[nicce]

I read the law and you are correct, there is explicitly mentioned that provider is not required to enforce seeking of CSAM evidence. However, they might be required to comply the demands of NCMEC if they ask to stop redistribution of certain visual depictions by providing hashes. This is were scanning steps in.

[GeekyBear]

> Then HN taught me that any company storing images on their infrastructure in the US must report pedophilic images to the US government.

It's certainly been going on for the past decade.

For example:

>a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account

https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...

[matheusmoreira]

Simple.

1. Encrypt everything.

2. Don't store images on your servers at all.

There's nothing to report if all you have is some encrypted blob. Alternatively, just don't consume any user data at all. Data is and should be a massive liability.

[skinkestek]

> Data is and should be a massive liability.

My thoughs as well.

If you don't want there very dangerous weapon you have thought out to be abused, don't create a physical assembly of it and don't tell anyone who has a habit of abusing powerful weapons.

[ianmiers]

Apple's banned image reporting wont stay iCloud only.iMessage is next. Maybe all data on your phone. 1) phone scanning is overkill for pics already on their servers. You don't build this and take the PR flack for something you can already do server side 2) Even if it's somehow not Apple's plan, they will be forced to use it on iMessage. Congress has been trying to for years.See the EARN IT act[0].

Apple just erroneously said "it's safe" despite the fact that it clearly can be abused.

[0] https://blog.cryptographyengineering.com/2020/03/06/earn-it-...

[zepto]

> You don't build this and take the PR flack for something you can already do server side

That’s exactly what you do if you plan to enable E2E.

[ianmiers]

Yep. That certainly is the next step. And then, once you are scanning encrypted data, iMessage is next whether you want it or not.

[nicce]

It is not the next step, it is already there, if you read the technical papers. Additional encryption level comes to iCloud images with this change, and Apple can’t see your photos anymore unless CSAM threshold is achieved.

[zepto]

> And then, once you are scanning encrypted data,

They aren’t.

> iMessage is next whether you want it or not.

Is there some evidence you have of this plan? Sounds like this is just a fear you have.

[ianmiers]

>Is there some evidence you have of this plan? Sounds like this is just a fear you have.

The EARN IT act. It may not be Apple's plan, Apple's plan, as you suggest, might only be for doing scanning on encrypted iCloud and excluding encrypted iMessage. But what Apple will be pushed to do after that is pretty clear.

[zepto]

If the government passes a law mandating that encrypted messages be scanned, it won’t be done using this CSAM mechanism, and it won’t only be Apple doing it.

In short, you might be right to be afraid of this outcome, but it has nothing whatsoever to do with CSAM countermeasures.

[thw0rted]

From everything that I've read, iCloud Photo Library is currently encrypted on the server, with a key that Apple only uses when presented with a warrant. If I ran the company (disclaimer: I do not) I'd implement this with an airgapped system in a vault somewhere, where a very small number of people have access to bring encrypted images in on a CD-R under two-person control.

That being said, one of two things is true. Either Apple does exactly what they say, in which case they are not able to perform server-side content / fingerprint scanning, or Apple is outright lying about only using their key on behalf of law enforcement. This latter case would open them to all sorts of legal liabilities, like a suit from shareholders for false reports. It would also require the silence of every Apple engineer who has ever been involved in at least their iCloud Photo program, and probably a bunch of server infrastructure as well. Additionally, they'd be legally obligated to report their scan results to the NCMEC but would have to do so in a way that doesn't give away that they're lying about how their systems work.

[hef19898]

Because once that functionality is there it affects everyone, not just in the US. And it basically means we sell out our democratic principles, or rather allow our tech giants to sell it out. Or force them to do it, like our elected governments doing it. Either way, I don't like the outcome.

[zepto]

> Because once that functionality is there it affects everyone, not just in the US.

The functionality to detect CSAM uploaded to Apple’s servers or sent to pre-teens?

> And it basically means we sell out our democratic principles

What democratic principle is being sold out?

[hef19898]

The right to secret communication. The right of not being under surveillance. The government cannot open letters without a warrant, but somehow Apple, Google, MS and co can sniff through electronic communication as they see fit because of a clause in an EULA. No idea how came there, but maybe the days when Stasi surveillance was the poster child of government intrusion into private life are too long gone to be remembered. Or they aren't and certain people choose to make shit load of money from the thing.

[jdavis703]

> The government cannot open letters without a warrant, but somehow Apple, Google, MS and co can sniff through electronic communication

This is no different than a private doctor testing for illicit drugs and reporting results to the DEA (they literally do this for ADHD patients.)

[pseudalopex]

I know American ADHD patients. None of them take drug tests.

[jdavis703]

I’m an American ADHD patient. My doctor made me (and his other patients) come in on random weekends for drug tests. He said the DEA made him report his records.

[amanaplanacanal]

Might as well make it legal for police to search our houses at will, as long as they are looking for child abuse images. Doesn’t sound much like the US any more at that point.

[zepto]

> Apple, … can sniff through electronic communication as they see fit

Except that they can’t and don’t.

[hef19898]

Google is checking, apparently, Gmail for cp. Apple is doing it, soon, on your phone. Checking your mail for analog cp requires a warrant and can only be done by police. See the difference?

[zepto]

> Apple is doing it, soon, on your phone.

Apple is only checking images you choose to upload to iCloud photos to see if you are uploading a collection of CSAM. This is entirely optional, and they have publicly explained what they are doing.

They are not sniffing through your communications as they see fit.

[sebzim4500]

They could still do the scanning, but if the photo fails it would just refuse to upload it and display an error to inform the user that the photo will not synchronize. There is no reason that the results of the scans need to be sent to Apple servers.

[nicce]

If you read the technical details, result of the scan is packed with the photo. So, if upload of photo fails, then result of the scan is not uploaded as well.

[tyingq]

>if you don't want to store pictures in clear on your servers

Reading https://support.apple.com/en-us/HT202303 , it seems that Apple may encrypt pictures on their servers, but they have the key. The list of what's actually end-to-end encrypted doesn't include photos. So, they may be scanning on your phone, but they can scan on their servers if they wanted to.

[thw0rted]

I posted more detail upthread but what I've found suggests that Apple does have a key to decrypt pictures but they claim to use it only to respond to a warrant. (They could of course be lying about that, but I don't believe they are.)

[siscia]

I believe the want this update exactly to enable E2E encryption.

In this way the can get rid of the keys on their servers and still find pedo pictures.

[nojito]

Apple doesn’t scan iCloud for CSAM and refuses to do it. Which is why they researched intensively on differential privacy.

[sneak]

But they could, as iCloud Photos is not e2e (Apple can read all of it) and they turn over the user data on over 30,000 users per year to the USG without even a warrant.

This is just farce.

[zepto]

> But they could, as iCloud Photos is not e2e

Client side scanning is a prerequsite to making it e2e if you also want countermeasures against CSAM.

[amanaplanacanal]

Has Apple said this is what they are going to do, or are people just guessing?

[zepto]

They haven’t announced this, but they invest a lot in encryption and privacy, and have stated that user privacy is a value of theirs. They have also expressed that they don’t want access to be able to be forced by law enforcement.

[sneak]

Their actions speak louder than their words.

[nojito]

No they refuse to indiscriminately scan iCloud.

[thw0rted]

Does that 30k number include iCloud Photo data? Do you have a citation for this?

[sneak]

Apple's own transparency report, under FISA orders. Presumably it includes all subscriber data they can access for the specified accounts, so likely contacts, photos, and device backups (full iMessage chat history, or sync keys to decrypt same).

FISA orders are not warrants and do not require probable cause; the FISA Amendments Act Section 702 spying that goes on (aka PRISM internally to the IC) pulls data directly from cloud provider systems without a search warrant and was cited by Ed Snowden as one of the main reasons he came forward.

[ryanlol]

> any company storing images on their infrastructure in the US must report pedophilic images to the US government

Ones they know of…

> What other technical approach are people advocating for?

Apple already has a technical solution, encryption.

[zepto]

> Apple already has a technical solution, encryption.

How does encryption help prevent porn being sent to pre-teens?

[ryanlol]

That’s a completely different feature than the one we’re discussing. These things were announced together, but they are not the same.

Nobody is objecting to opt-in clientside content filtering.

[zepto]

Both of the features involve opt-in client side content filtering.

The only objections are to that.

[ryanlol]

The CSAM scanning is not opt-in.

Sure, you could stop using iCloud. That’s opt-out.

[zepto]

That’s not correct. This applies only to iCloud Photo Library, not to iCloud as a whole.

iCloud Photo Library is an optional feature, and there are numerous alternatives.

[ashildr]

Encryption does not help, Apple still is responsible. If Apple intends to let the user store photos in iCloud (or send by imessage) encrypted, they either have to keep the keys, so they can decrypt and scan the photos or or to keep the user from uploading incriminating content. Apple found a third way: they will only get to reconstruct the keys if the user uploads too many pictures triggering alarms.

[adambatkin]

Source? I am not aware of a law in the US that requires Apple to actively scan images, or to store them unencrypted (or keep copies of the keys).

[hef19898]

The US aren't the only government with a stake in that. And countries like China, Saudi, the Emirates have a lot of leverage. Financially and diplomatic. Heck, Facebook bowed to Myanmar just to get the users there.

[nicce]

Every cloud infrastructure holder is required for doing that. Closing an eye does not take a duty away. You must be actively pursuing that. Encryption would start flood of new laws

https://www.govinfo.gov/app/details/USCODE-2011-title18/USCO...

[skinkestek]

Tarsnap exists so either it is legal when done right or tarsnap is a walking dead and I haven't heard anything to that effect from any credible source.

[nicce]

I guess that service slightly goes out of the scope for active scanning, because it is for general backup, not a cloud especially for photo sharing and storing.

[ryanlol]

Those laws do not exist (yet?). You can’t justify this as a compliance measure for legislation that does not exist.

[nicce]

Yes, but current laws also restrict storing images as E2E encrypted, so there is dilemma?

[dhfjskh]

>Then HN taught me that any company storing images on their infrastructure in the US must report pedophilic images to the US government.

That's not true though.

[PolCPP]

You could generate hash on device, send it to server alongside the file, once validated delete hash or create decoding voucher.

[nicce]

This is exactly what they are doing right now. Decoding voucher applies when their system thinks that too many hashes goes into CSAM category.

[starfallg]

>What other technical approach are people advocating for?

Reduce user data stored in cloud data centres as much as possible. This is the approach taken by Whatsapp, so not surprised they are the ones most vocal against it.

And at the risk of appearing to be supportive of a Facebook product, I think this is the right way to take computing. We don't need a central place to put stuff or to do compute when we can do it on our own devices. We just need orchestration.

[nicce]

It is a bit ironical that WhatsApp is worried about privacy. All message metadata is unecrypted and part of their business model. They don’t know about message contents, but they know everything about your social network (who do you message and when, who are part of your groups etc.) Add cross-app tracking with Facebook APIs and soon they can also categorize your message contents.

[asutekku]

How would you do it? With whatsapp you have the file on messaging partner’s phone. People do not want to share their images over a peer network with random people.

[LatteLazy]

Apple aren't required to decrypt anything. This is why ever other server/storage provider are not also demanding access to everything client side (or keys to decrypt server side). It's a red herring for Apple to pretend they're "required" to do this, they're no more required to do so any more than the post office are required to open your mail on the off chance they might be handling CP...