[fraa-orolo]

A false positive in matching faces results in a click to fix it or a wrongly categorized photo. A false positive in this new thing may land you in jail or have your life destroyed. Even an allegation of something so heinous is enough to ruin a life.

The "one in trillion" chance of false positives is Apple's invention. They haven't scanned trillions of photos and it's a guess. And you need multiple false positives, yet no one says how many, so it could be a low number. Either way, even with how small the chance of it being wrong is, the consequences for the individual are catastrophic. No one sane should accept that kind of risk/reward ratio.

"Oh, and one more thing, and we think you'll love it. You can back up your entire camera roll for just $10 a month and a really infinitesimally minuscule chance that you and your family will be completely disgraced in the public eye, and you'll get raped and murdered in prison for nothing."

[systoll]

Ok.

So iCloud Photos circa 2020 [and Google Photos and Facebook and Dropbox and OneDrive] aren’t a risk you should be willing to take.

This feature doesn’t change anything in that regard; the scanning was already happening.

[XorNot]

I literally do not take that risk in 2021. I do, currently, make the reasoned assurance that the computational overhead of pushing changes down to my phone, and the general international security community, are keeping me approximately abreast of whether my private device is actively spying on me (short answer: it definitely is, longer answer: but to what specific intent?)

Apple's new policy is: "of course your phone is scanning and flagging your private files to our server - that's normal behavior! Don't worry about it".