I feel like this is a false sense of security. Even before this change, they can easily access and scan photos on your device. If they do any post-processing of the image on device, they already do.
It’s not a false sense of security, it’s a clear delimitation between theirs and mine; Debian package maintainers can also slip a scanner on your machine but that is a big line to cross on purpose and without notifying the user.
That is technically true but in a real very practical sense everyone here using OSS absolutely is trusting a third party because they are not auditing every bit of code they run. For less technical people there is effectively zero difference between open and closed software.
It’s really disingenuous to suggest that open source isn’t dependent on trust, you just change who you are trusting. Even if the case is someone else is auditing that code, you’re trusting that person instead of the repository owners.
I’ll concede that at least that possibility to audit exists but personally I do have to trust to a certain extent that third parties aren’t trying to fuck me over.
> Even if the case is someone else is auditing that code, you’re trusting that person instead of the repository owners.
Suppose Debian's dev process happened at monthly in-person meetings where minutes were taken and a new snapshot of the OS (without any specific attribution) released.
If that were the case, I'd rankly speculate that Debian devs would have misrepresented what happened in the openssl debacle. A claim would have been made that some openssl dev was present and signed off on the change. That dev would have then made a counterclaim that regular procedure wasn't followed, to which another dev would claim it was the openssl representative's responsibility to call for a review of relevant changes in the breakout session of day three before the second vote for the fourth day's schedule of changes to be finalized.
Instead, there is a very public history of events that led up to the debacle that anyone can consult. That distinction is important-- it means that once trust is in question, anyone-- including me-- can pile on and view a museum of the debacle to determine exactly how awful Debian's policy was wrt security-related changes.
There is no such museum for proprietary software, and that is a big deal.
That's certainly true, and it is a strong 'selling point,' so to speak, for open software. But openness is just one feature of many that people use for making considerations about the sort of software they run and frankly, for an average consumer, it probably weighs extremely low on their scale, because in either case it's effectively a black box, where having access to that information doesn't actually make them more informed, nor do they necessarily care to be informed.
Most people don't care to follow the controversies of tech unless it becomes a tremendously big issue, but even then, as we've seen here, there are plenty of people that simply don't have the technical acumen to really do any meaningful analysis of what's being presented to them and are depending on others to form their opinion, whether that be a friend/family member or some tech pundit writing an article on a major news organization's website.
Trusting Apple presents a risk to consumers but I'd argue that for many consumers, this has been a reasonable risk to take to date. This recent announcement is changing that risk factor significantly, though in the end it may still end up being a worthwhile one for a lot of people. Open Source isn't the be all end all solution to this, as great as that'd be.
Thinking about this.. I guess my trust, is that someone smarter than I will notice it, cause a fuss, and the community will raise pitch forks and.. git forks. My trust is in the community, I hope it can stay healthy and diverse for all time.
Trusting in a group of people like you to cover areas you might not is the benefit of open source and a healthy community.
With Apple you have to trust them and trust they don't get national security order.
I trust that if everyone who had the ability to audit OSS got a national security order it would leak and it would be impossible for many who live in other nations.
Maybe if you drink from the NPM PyPI firehouse without checking (as too many do unfortunately).
For regular Linux distribution there are maintainers updating packages from upstream source that can spot malicious changes slipped in upstream. And if maintainers in one district don't notice, it is likely some in onether distro will.
And there are LTS/enterprise distros where upstream changes take much longer to get in and the distro does not change much after release. Making it even less likely a sudden malicious change will get in unnoticed.
Somewhere along the line someone is producing and signing the binaries that find their way onto their computer, they could produce those binaries from different source code and I would be none the wiser.
Debian tries to be reproducible, so to avoid being caught they might need to control the mirror to so that they could send it to only me. I.e. if I'm lucky it would take a total of 2 people to put malicious binaries on my computer (1 with a signing key, 1 with access to the mirror I download things from).
The only way what you said is not true for any networked device is to just go down to the river and throw it in and never use a digital device again. It's not a false sense of security, it's a calculated position on security and what you will accept, moving spying from the server to the phone was the last straw for a lot of people.
Apple already scans your photos for faces and syncs found faces through iCloud. I’d imagine updating that machine learning model is at least as straightforward as this one.
They're searching for different things though. To my knowledge, before now iOS has never scanned for fingerprints of specific photographs. It would be so darn easy to replace the CSAM database with fingerprints of known tiananmen square photos...
That is a distinction without a difference. I’m sure you could put together quite a good tank man classifier (proof: Google Reverse Image Search works quite well), and it’d catch variations which a perceptual hash wouldn’t.
The only difference is intent. The technical risk has not changed at all.
The technical risk to user privacy - if your threat model is a coerced Apple building surveillance features for nation state actors - is exactly the same between CSAM detection and Photos intelligence which sync results through iCloud. In fact, the latter is more generalizable, has no threshold protections, and so is likely worse.
It's the legal risk that is the biggest problem here. Now that every politician out there knows that this can be done for child porn, there'll be plenty demanding the same for other stuff. And this puts Apple in a rather difficult position, since, with every such demand, they have to either accede, or explain why it's not "important enough" - which then is easily weaponized to bash them.
And not just Apple. Once technical feasibility is proven, I can easily see governments mandating this scheme for all devices sold. At that point, it can get even more ugly, since e.g. custom ROMs and such could be seen as a loophole, and cracked down upon.
Nonsense. Building an entire system as opposed to adding a single image to a database is a substantially different level of effort. In the US at least this was used successfully as a defense. The US cannot coerce companies build new things on their behalf because it would effectively create "forced speech" which is forbidden by the US Constitution. However they can be coerced if there is minimal effort like adding a single hash to a database.
Exactly this. The whole thing is a red herring. If Apple wanted to go evil, they can easily do so, and this very complex CSAM mechanism is the last thing that will help them.
A false positive in matching faces results in a click to fix it or a wrongly categorized photo. A false positive in this new thing may land you in jail or have your life destroyed. Even an allegation of something so heinous is enough to ruin a life.
The "one in trillion" chance of false positives is Apple's invention. They haven't scanned trillions of photos and it's a guess. And you need multiple false positives, yet no one says how many, so it could be a low number. Either way, even with how small the chance of it being wrong is, the consequences for the individual are catastrophic. No one sane should accept that kind of risk/reward ratio.
"Oh, and one more thing, and we think you'll love it. You can back up your entire camera roll for just $10 a month and a really infinitesimally minuscule chance that you and your family will be completely disgraced in the public eye, and you'll get raped and murdered in prison for nothing."
I literally do not take that risk in 2021. I do, currently, make the reasoned assurance that the computational overhead of pushing changes down to my phone, and the general international security community, are keeping me approximately abreast of whether my private device is actively spying on me (short answer: it definitely is, longer answer: but to what specific intent?)
Apple's new policy is: "of course your phone is scanning and flagging your private files to our server - that's normal behavior! Don't worry about it".