[kergonath]

It’s very muddy, though. The number of pictures on one’s hard drive is irrelevant to the fact that a child has been abused or not. In your example, none of the children would have been abused. Also, who gets to define how much “a lot” is? We can’t say that it’s ok if it’s your children (or grand-children’s, or nephews, etc), because most of child abuse cases involve close family members or close friends.

We definitely should punish exploitation of children, including sexual, and we definitely should punish distributing images of this. But conflating exploiting, distributing, and viewing, and then putting a big taboo on this, is really not ideal. These things are different. Otherwise what we end up with is righteous frenzy when someone gets punished for sexting.

[shakna]

That is correct. The children within the images that are classified as CSAM don't necessarily have (though frequently are) to be abused.

For example, the NCMEC database contains hashes for Nirvana's Nevermind cover. Completely innocent to possess within it's original and intended context.

I have not said whether I agree with this, because I do see problems when automating the process.

However, the precedent for it being used as a flag for law enforcement has already happened. Having a collection of similar imagery is considered CSAM - and that is likely correct. A collection is probably not innocent. But having one or two images may happen incidentally without your awareness of it.

As to who decides what constitutes significance? That is where you'll hit the most problems, and reasonable discussion of it will be quickly shut down with the same arguments used for automating a flagging system. The conversation requires nuance, but those currently calling for such systems aren't interested in a good faith discussion.

[kergonath]

> I have not said whether I agree with this, because I do see problems when automating the process.

The process will never be fully automated, regardless of what they say. Cases will need to be reviewed. Things will need to be checked at some point.

They are trying to play the cog in the machine, that mechanically transmits information to law enforcement. But if we’ve learnt anything the last decade is that cogs are not impartial and can be very dangerous, if only because of the scale at which they operate. I can see several ways a user can face a kafkaesque uphill battle to prove their innocence. In several countries, just a child pornography case can be a social death sentence. And even a fraction of a percent of mistakes will mean millions of people might be dragged in this (let’s not kid ourselves: this is never going to stay in the US).

Personally I think (what Apple is doing) is misguided and ripe for abuse. I am very disappointed that they, of all companies, are pushing this nightmare.

> As to who decides what constitutes significance? That is where you'll hit the most problems, and reasonable discussion of it will be quickly shut down with the same arguments used for automating a flagging system. The conversation requires nuance, but those currently calling for such systems aren't interested in a good faith discussion.

Ultimately, a tech company has no business making this sort of decisions. This is something that needs to be sorted out by law.

Unfortunately, a nuanced discussion is very unlikely these days. Anyone looking not agressive enough will be pilloried.