[larschdk]

5. The system can easily be abused by governments or malicious actors to frame innocent people. People merely suspected of keeping such images are de-facto punished and stripped of rights even without standing before a judge or getting a conviction.

[newbamboo]

This is my primary concern. It will become a weapon to destroy the lives of anyone who is targeted by someone with middling or better hacking skills. A sort of digital “swatting” that makes using apple products a no go for anyone with cyber-enemies (one can’t opt out of Apple ID and apply security updates).

[GeekyBear]

Google has been scanning your entire account for kiddie porn for the past decade.

>a man [was] arrested on child pornography charges, after Google tipped off authorities about illegal images found in the Houston suspect's Gmail account

https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...

Their system can easily be abused by governments or malicious actors to frame innocent people.

[Unklejoe]

Once again, this scan takes place on _their_ servers on data that is stored on _their_ servers. It does not take place on the device itself, which is the case with this new Apple thing.

[GeekyBear]

Once again, the notion that everything on the device is scanned under Apple's system was never true.

Only photos you attempt to upload to Apple's iCloud are scanned. If you turn off iCloud photos, NOTHING is scanned.

>Q: So if iCloud Photos is disabled, the system does not work, which is the public language in the FAQ. I just wanted to ask specifically, when you disable iCloud Photos, does this system continue to create hashes of your photos on device, or is it completely inactive at that point?

A: If users are not using iCloud Photos, NeuralHash will not run

https://techcrunch.com/2021/08/10/interview-apples-head-of-p...

[Unklejoe]

> Once again, the notion that everything on the device is scanned under Apple's system was never true.

That isn't what I said.

Also, that's not why most people are so upset. Most people are so upset mainly because Apple has now proven that the capability exists, so they can now be more easily compelled by governments to scan for "extra things".

Prior to this, if a government asked Apple to scan someone's phone, Apple could respond with "we don't have that capability", and it would presumably be a tough legal battle to force a company to add a capability that doesn't exist.

This hurdle is now much lower. The effort has gone from "force Apple to design a new system for scanning phones" to "add these couple of hashes to the pre-existing database".

Also, expanding this from just iCloud upload candidates to the entire device is a very small leap now. I mean, the bad guys could just turn off iCloud, and we must think of the children...

Then you have Apple's "reassurance" that they won't comply with government requests to scan for additional things, which is completely moot considering Apple relies on a third party database and has absolutely no control or idea of what the hashes really are.

[GeekyBear]

The notion that scanning cloud data on device is somehow worse than doing the same thing on server is deeply flawed.

If you have a false positive on device, nothing is sent to Apple's servers. It takes several (possibly false) positives at once to trigger a human review.

If you have a single false positive on server, that data is sitting there where it can be subpoenaed and abused.

Also, recent history shows that Apple is willing to fight government demands to invade user privacy in court.

[shawnz]

> Also, recent history shows that Apple is willing to fight government demands to invade user privacy in court.

I can only think of one instance where they did that (the San Bernardino shooter case), and the request was hugely overreaching (the FBI wanted them to compromise their software update signing services), and also they actually DID comply with giving the FBI access to their iCloud data -- just not the software update service.

In fact this report suggests that Apple cooperating with the FBI when it comes to subpoenaing iCloud data is nothing new: https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...

[lucasyvas]

This fact is well known and changes nothing. The problem is that the system exists at all. The fine print WILL change - it always does, and that's also a well known fact.

[GeekyBear]

By that logic, Google will definitely begin selling their treasure trove of user data to anyone with a checkbook, because the fine print WILL change.

[vharuck]

Yes, that's a very reasonable assumption. I assume all information I give out will be shared beyond my control, unless the recipient promises in writing to protect it and would suffer proportionally if they broke that promise. In practice, this only happens when federal regulations apply (i.e., health care or banking).

If you want to rely on other people behaving a certain way in the future, either form a personal relationship or write up a contract.

[lucasyvas]

It's completely within the realm of possibility that this could happen. A few bad quarters down the road and a leadership change might be all it takes.

Many of us are taking the perspective of decades long changes given our current trajectory.

If not in our time, it could be in our children's time. This is an extremely dangerous system.

[belorn]

Google will maximize the method to monetize user data. They have done that in the past, they will continue to do so in the future.

Collect data and monetize it. That is what google is. They don't provide free email or analytic software out of the goodness of their heart.

[csmiller]

It’s a difference in policy vs technical capability. Currently the policy is only scan when iCloud Photos is enabled, but the capability to scan at any/all times is just a policy change away.

[GeekyBear]

No, it's a difference between scanning the files that users store in their respective clouds on-server or on-device.

Scanning on-device (where a single false positive cannot be subpoenaed and misused to incriminate their customers) is simply more private.

>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime

https://www.dailymail.co.uk/news/article-7897319/Police-arre...

[adventured]

Apple's new system is scanning personal property that doesn't belong to them and isn't yet in their cloud.

Gmail files that get scanned are contained on Google's property, in their cloud, on their machines.

Entirely different context.

It's the difference between the USPS coming into my home without permission and going through my documents, records, mail - versus if I send mail through their system and they track it, scan the envelope, etc.

The iPhone used to be pretty obviously personal property, now Apple is saying that's clearly no longer going to be the case going forward.

[crusty]

Oh, I like this USPS analogy but I'll clean it up. Google photos and chat are like a USPS that only stores and transmits post cards. It's understood by the creator/sender that anyone who has access to them can read them. Apple here is a USPS that sends sealed envelopes. They (say they) can't read what's inside as it's sent or stored. With this change they will create the 'capability' to show up whenever you decide to send an envelope and read it before you seal it up for sending.

Meh, nevermind. That's not much cleaner.

[GeekyBear]

> Apple's new system is scanning personal property that doesn't belong to them and isn't yet in their cloud.

Apple's new system only scans photos you attempt to upload to their cloud.

Nothing else is scanned.

Scanning the files on server, the way Google and Microsoft do it, means that false positive data is lying around where it can be subpoenaed and used to incriminate innocent people.

>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime

https://www.dailymail.co.uk/news/article-7897319/Police-arre...

[_-david-_]

>Apple's new system only scans photos you attempt to upload to their cloud.

And what if in the future they decide they need to scan more than images going to the cloud? What if there is some huge epidemic of child abuse or some other terrible thing and Apple decides they need to do more?

Once you open Pandora's Box you can't close it.

[GeekyBear]

What if in the future Google starts selling your location data to anyone willing to write them a check?

Once you open the Pandora's Box of collecting location data, you can't close it.

[grupthink]

Both will happen. Basically, Apple is able to scan data on your phone, and Google is able to scan data on their servers.

[arvinsim]

It doesn't matter if the scan is conditional.

It matters that the capability is there.

[nicce]

Capability has always been there. It is only worded now in a way which made it the most people understand. Speculation about "doing something in hidden", is as valid as before.

In reality, we can be only be mad when they they are publicly making things worse in black box systems. Not about something, which is "policy change" away. Let's be mad when they actually change that policy.

[strofcon]

Your whatboutism is profoundly unhelpful here.

Yes, other companies are doing bad things, and they should be stopped.

Doesn't by any stretch of the imagination mean that Apple should be allowed to do something even worse.

[GeekyBear]

Other companies aren't "doing bad things" they are handling scanning the contents of their cloud services in a much more user hostile way.

Keeping that data on their server means it can be subpoenaed and misused.

[strofcon]

I mean, we can split hairs over the words to use, but ultimately "immoral and unethical things are being done by big companies that hold all your stuff". The sentiment is the same.

What I'm getting at is that the things Google and Microsoft are doing are entirely irrelevant to the conversation at hand.

Apple is going to compromise your device's privacy in the name of child safety, and will - invariably - eventually cave to pressure to extend that capability well beyond it's originally well-meaning use case.

Stop bringing up what other companies are doing - it is, as I said, entirely irrelevant.