[Shaddox]

This quote sums it up perfectly :

“Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies.

The robber baron's cruelty may sometimes sleep ,his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience.

They may be more likely to go to Heaven yet at the same time likelier to make a Hell of earth. This very kindness stings with intolerable insult. To be "cured" against one's will and cured of states which we may not regard as disease is to be put on a level of those who have not yet reached the age of reason or those who never will; to be classed with infants, imbeciles, and domestic animals.” C.S. Lewis

There are so many problems with this feature :

- it can easily be used to identify also other types of dangerous material such as warnings against government activity or posters of organized protests

- some malicious actor can send you this kind of content in order to get you in trouble

- false positives

- moral busy bodying (your employer not agreeing with you going drinking sunday night)

Honestly, it feels like we're about to enter a new age of surveillance. Client-side surveillance.

[gambiting]

The consequences seem very simple - once this system is in place, it's only a matter of time until a state actor, be it China or US or Russia or pretty much anywhere goes to Apple and says "hey this hash matching algorithm you have there? If you want to keep operating in our country, you need to match <those> hashes too, no, we won't tell you what they are and what they represent, but you have to report to our state agency who had those present on the device".

Once the technology exists it will be abused.

[matheusmoreira]

Completely agree. I have absolutely no doubt this technology will be abused. I bet it will be used to silence dissent, opposition and wrongthink more often than to protect children.

[candiodari]

Why doesn't anyone ever question their intent to protect children? There is no such intent. When they do protect children, these are the sorts of observations of what happens when the state has 100% control over children:

https://www.kansascity.com/news/special-reports/article23820...

Why does the state, when they treat children like this, get to proceed on "more" child protection to protect children? Are we totally mad? Any effort to protect children is always a de facto effort to deliver more children into this system, and the state system is much worse than being abused in society (never mind that someone found there's actually more abuse in foster care than with abusive parents! Therefore even if you assume social services is always right ... they never protect kids against social services itself. Therefore you can reasonably assume that a kid that is getting abused will be forced into a worse situation by state "help")?

I mean if you want to protect children, obviously the first step is to fix the child protection system that everybody knows is badly broken (you constantly hear stories about schools sabotaging research into abusive parents to protect the child against social services, covering for the child, lying about attendance or even wounds, ..., because they know social services will be much worse for the child).

There's states where the corrections department provides better care (and definitely better educational instruction) for children than social services do. And yes: that is most definitely NOT because the corrections department provides quality instruction. It's just MUCH worse with social services.

They cover themselves by showing the worst possible situations in society "demonstrating the need to intervene". And you won't hear them talk about how they treat children ... because frankly everyone knows.

Reality is that research keeps finding the same thing: a child that gets abused at home ... is being treated better (gets a better future, better schools, better treatment, yes really, more to read, ...) than children "protected" from abuse by the state:

https://www.aeaweb.org/articles?id=10.1257/aer.97.5.1583

(and that's ignoring that reasons for placement are almost never that the child is unsafe. The number 1 reason is non-cooperation with mental health and/or social care by either one of the parents or the child themselves. There is not even an allegation that the child is unsafe and needs protection. Proof is never provided, because for child protection there is no required standard of proof in law)

And we're to believe that people who refuse to fix child services ... want extra power "to protect children"? How is this reasonable at all? They have proven they have no interest whatsoever in protecting children, as the cost cutting proves time and time again.

[still_grokking]

To add on this: Nothing else is to expect form a country that—under protection of its legal system!—tortures mentally ill and disabled children.

https://en.wikipedia.org/wiki/Judge_Rotenberg_Educational_Ce...

https://www.independent.co.uk/news/world/americas/massachuse...

[throwavocado]

I am supportive of Apple’s new child protection policies. I spent two years helping to build some of the tools used for scanning and reporting content (EDIT: not at Apple). There are a few sentiments in this thread that I'd like to address.

> Yeah, and none of these assholes (pardon the language) is willing to spend a penny to provide for millions of children suffering of poverty: free education, food, health, parental support. Nothing, zero, zilch. And these millions are suffering right now, this second, with life-long physical and psychological traumas that will perpetuate this poverty spiral forever.

Many people in child safety and I, personally, strongly support policies that enhance the lives of children, including improved education, food, access to health services, and support for parents. To your point, though, it's also true that many political leaders who vocally address the issue of child sexual abuse on the internet also happen to be opposed to the policies I would support. Like most political alliances, it is an uncomfortable one.

> Anyone questions you, just destroy their reputation by calling them a pedophile.

I see this sentiment a lot. I have worked with people across law enforcement, survivors, survivor advocates, NGOs, social workers, private companies, etc. and I don't know anyone who responds this way to people raising privacy concerns. At worst, they might, rightly or wrongly, consider you alarmist or uninformed or privileged (in that you don't have images of your abuse being actively traded on the internet). But a pedophile? I just can't imagine anyone I've worked with in this space accusing you or even thinking of you as a potential pedophile just because you're opposed to content scanning or want E2EE enabled, etc. I suppose maybe someone far, far removed from actual front line workers would say something so ridiculous.

---

Separately, I want to suggest that there are paths forward here that could include risk controls to reduce the risk that this technology gets extended beyond its initial purpose. Maybe NCMEC could provide verifiable digests of the lists used on your device to verify that additional things haven't been added to it. Or there could be public, concrete, transparent criteria for how and when a lead is referred for law enforcement action. By designing this system by which the matching occurs on device against a list that can be made available to the user, Apple has made content scanning far more privacy-preserving and also created avenues by which it could be further regulated and transparent. I'm very excited about it and, honestly, I think even the staunchest privacy advocates should be cautiously optimistic because it is, in my opinion, a step in the direction of less visibility into user data.

I think that privacy advocates are arguing in good faith for protecting society from a surveillance state. I think that advocates of scanning are arguing in good faith for protecting children. I also think that both sides are using language (screeching on the NCMEC side, comparisons to hostile regimes on the other) that make it very hard to move forward. This isn't pedophile privacy advocates vs. surveillance state NCMEC. Neither of those groups even exist. It's concerned citizens wanting freedom for all people vs. concerned citizens wanting freedom for all people.

[matheusmoreira]

HN is the only place I feel safe enough to post these pro-privacy opinions. In many other communities, I've seen people being accused of serious crimes for daring to take issue with whatever surveillance the authorities are trying to implement. I've seen people openly speculating about the CSAM stashes of encryption users. What else could they possibly be hiding, right?

I don't think I trust actual authorities either.

[Mindwipe]

Given NCMEC's continuing attacks against consenting adult sex workers, that they never seem to regard adult sexual identities as valid under any circumstances, that they repeatedly retweet abolitionist groups for adult sexual expression, their actions during Backpage, them lying to Congress about the level of abuse that occurs and the recent statements by their staff, I find it kind of baffling that anyone would defend their leadership at this point.

NCMEC are willing to completely compromise their mission in order to chase other moral crusades that are unrelated to children, and seem to never care about the consequences of any of the things they call for.

I don't trust them, and they've earned that.

[candiodari]

> Many people in child safety and I, personally, strongly support policies that enhance the lives of children, including improved education, food, access to health services, and support for parents. To your point, though, it's also true that ...

You work with people who, almost always against the will of children and parents kidnap children (sorry, but search on Youtube for images of their action: kidnap is the only correct term) ... then proceed to NOT care for those children and destroy their lives. Obviously this is the only correct reasoning is that this is entirely immoral until the care system is fixed, because they are not improving the lives of children, and their complete lack of caring about this tells you what they're goals aren't.

Watch "Short Term 12" for what facilities these children are provided with. Stop pretending that helping these people acquire children (because that's exactly what you're doing) helps a single child. It doesn't. Terrible, abusive, violent, parents take better care of children than these places do. The moral reaction should be to do what most of society, thankfully, does: sabotage the efforts of social workers.

And if you're unwilling to accept this, as soon as this whole covid thing dies down a bit, find the nearest children's facility and visit. Make sure to talk to the children. You will find you're making a big mistake.

Whatever you tell yourself, please don't believe that you're helping children. You're not. You're helping to destroy their lives

> I see this sentiment a lot. I have worked with people across law enforcement, survivors, survivor advocates, NGOs, social workers, private companies, etc. and I don't know anyone who responds this way to people raising privacy concerns.

I'm adding this response to the other reply to your comment ... which makes 2 people who disagree with your assessment: you are likely to be threatened in many places. I feel like one might even say it's likely we're 2 people who have been threatened.

I would like to add that either as a child or an adult, child protection authorities, the people you help, will threaten you, and I've never known a single exception if they think (correctly or otherwise) that you're hiding something from them. That's if you're at their mercy (which is why every child in child services makes sure to commit some despicable/violent/minor criminal act or two every month and keep it secret: if you don't have something to confess that won't get you sent to a "secure facility" you will be horribly punished at some unexpected random time. That's how these people work. And over time you may learn that, for a kid in the system, a whole host of places are traps. Like the hospital, child services itself, police, homeless shelters or school. As in every person in one of those places will be shown your "history" as soon as you mention your name and if they report you ... very bad things will happen. Some children explicitly make bad things happen (e.g. commit violent theft), because they'll get sent to "juvie" and finally the stress of suddenly getting punished out of nowhere disappears. Also some believe you get better schooling in juvie. So you hide, even if you're hurt or sick. This is why some of those kids respond angrily or even violently if someone suggests they should see a doctor for whatever reason. Sometimes literally to make sure the option of suicide remains open (which is difficult in "secure" care). And why does this happen? NOT to protect children: to protect themselves and "their reputation", and their peace of mind against these children).

This, of course, you will never hear your new friends mention needs fixing. They are in fact fighting with this side of the system over money, so that EVEN LESS money goes to the kids the system takes care of. That, too, you will never hear from them. They are doing the opposite of what someone who means well with disadvantaged children will do.

I have zero doubts they will use your work, not to convict offenders, because that's hard, years of very difficult work, but to bring more kids into a system that destroys children's lives, and MORE so than the worst of parents do. Because throwing children into this system (and destroying their lives) is very easy. I'm sure occasionally they will convict an offender (which, incidentally, doesn't help the kids) or get a good outcome for a child. It happens. If is absolutely not common.

And not to worry they will put great emphasis on this statement: "I've never seen anyone in the system who didn't mean well". Most are idiots, by the way, if you keep digging at their motivations, they will reveal themselves very clearly quite quickly.

These "people in child safety" DO NOT mean well with children. They merely hate a portion of society (which includes those victimized children) and want to see them punished HARD. If you are a moral person, volunteer at a nearby children's shelter and show understanding when the inevitable happens and you get taken advantage of (do not worry, you will learn). DO NOT HELP THESE PEOPLE GET MORE CHILDREN.

[still_grokking]

Meta-question: Is there a way to get rid of such "hot air" comments form throwaway accounts?

This comment does not add any argument in favor of its point, despite its length.

[sharken]

And i think if you're working on these systems, then it's all too easy to only think about the happy scenario.

But time has shown that an initial good concept will transform into something worse.

We have a very recent example with Corona contact-tracing apps, that law enforcement in multiple democratic countries are now using for other purposes.

So no, we should not allow this client-side scanning to go through, it will not end well.

[bsenftner]

This is what will happen, it's obvious to anyone not emotionally invested in the propaganda theater driving all social interaction.

[nyx-aiur]

This is such a stupid fallacy and it comes up every time something like this is discussed. You don't know if anything is or will be abused you expect it because you expect your elected government to do so. The problem here is with you or your electors not with the technology itself. You want to fix the symptoms but not the problem. As usual.

[gambiting]

How so? The UK's snooper's charter that compels ISPs to save your entire browsing history for a year was only meant to be used to catch pedophiles and terrorists, and now 17 different agencies have been given completely warrantless access to the database, including the UK's Food Standards Agency(??!?!?!?). People have already been fired for abusing access too, so it definitely happens too.

>>The problem here is with you or your electors not with the technology itself

No offense, but this is such a shitty answer, and it's always made by apologists for these incredibly invasive technologies. Like, please explain to me, in simple terms, how can I, an immigrant who doesn't even have the ability to vote, vote to make an American corporation do nor not do something. I'm super curious.

[nyx-aiur]

It's the exact same shit I was talking about instead of fixing your governing body you want to fix it with technology. All the crypto apologist are the same way if the government YOU are electing does something stupid you want to fix the symptom and not the governing body. You are just throwing around goal posts instead of working on the real problem.

[nemothekid]

>government YOU are electing does something stupid

Is America the entire world to you? What should a Chinese citizen do? What should a Saudi citizen do? What should a Russian citizen do? Even if you ignore the fact that the chance of fascism in American is not zero, why should Apple make it easier for totalitarian regimes to spy on their citizens? Or do you expect a Saudi person to "just move to America"?

[nyx-aiur]

I am talking to people on this website which is banned in 3 of the 4 countries you are talking about. Stop moving your shitty goalpost we are talking about the US, Europe and other democracies. You know what would help people in regimes? Governing bodies that stand up for them in other countries... guess who could change that.

[gambiting]

>>if the government YOU are electing

Except like I clearly said above, I'm an immigrant without the right to vote, so I'm not electing shit. Again, how exactly am I supposed to vote my way out of this?

[matheusmoreira]

Civil disobedience. If we think a law is unjust, it is our duty to disobey and undermine it. Technology is a tool that allows us to do exactly that.

[matheusmoreira]

> You don't know if anything is or will be abused

Actually I do. Governments abuse their powers all the time. They have done it before, are doing it right now and will continue to do it in the future. This is not fallacy, it is fact.

Here's an example:

https://en.wikipedia.org/wiki/LOVEINT

The only solution is to take their power away. No way to abuse power they don't have. We must make it technologically impossible for them to spy on us.

[nyx-aiur]

That's the exact same fallacy you are proposing to fix a symptom and not the problem. You want to reign in the government YOU ARE ELECTING TO GOVERN you with technology instead of real political work. Either vote different or get into politics and fix yourself.

Your wikipedia link doesn't show anything regarding abuse of the governing body ALL of the examples are from private persons.

[gambiting]

>>Your wikipedia link doesn't show anything regarding abuse of the governing body ALL of the examples are from private persons.

Have you even like....read the page they linked?

"Siobhan Gorman (2013-08-23). "NSA Officers Spy on Love Interests". Washington Wire. The Wallstreet Journal. "

Are NSA Officers "private persons" now? They are government employees, they were abusing the power they were given while employed by the government. It doesn't matter in the slightest if they were abusing the power for private or state gain, it's a state agency and its employees abusing the access, that implicitly makes it the state abusing the power they have.

[nyx-aiur]

Wow if you can't distinguish between rogue agents and an institutional abuse of power there is nothing left to argue.

[matheusmoreira]

I didn't elect anyone. There is not a single politician in power right now that represents me. I'm not even american to begin with so it's not like I have any influence over american administration.

In any case, there's no reason why politics and democracy ought to be the only way to bring about change. We have a far more powerful tool: technology.

Governments make their laws. People make technology that neutralizes their laws. They make new laws. People make new technology. And so on. With every iteration, the government must become ever more tyrannical in order to maintain the same level of control over the population it previously enjoyed. If this loop continues long enough, we'll either end up with an uncontrollable population or with an oppressive totalitarian state. Hopefully limits will be found along the way.

> Your wikipedia link doesn't show anything regarding abuse of the governing body ALL of the examples are from private persons.

A government employee abused his access to the USA's warrantless surveillance apparatus in order to spy on his loved ones. If this isn't abuse of power, I don't know what is.

Honestly, it's just human nature. No person should ever be given such powers to begin with. I wouldn't trust myself with such power. It should be impossible to spy on everyone.

[raxxorrax]

Any modern state implements separation of powers, trias politica in most cases. Your argument ignores that. You also haven't made clear what fallacy you mean.

Want to fix child abuse? Fund teachers and child care. Apple cannot help those kids and I don't mean that as an indignation towards the company.

[srmarm]

The thing is tools like this will only make it harder to fix the problem.

[IntrepidWorm]

It's not just idle speculation- history speaks for itself.