[franga2000]

> Just how stupid do you think lawmakers, judges, prosecutors, and police are?

Very! Unimaginably so! A friend of mine from Germany received a GIF that contained ONE FRAME of CSAM from someone in a group chat, Whatsapp auto-downloaded it into the gallery, something auto reported it and a month later, cops showed up to take away all his electronic devices. This is apparently a thing people do there, like americans SWAT livestreamers. I think it took over a year for them to return his devices. He had to pay for a lawyer and buy a new phone and laptop. He wasn't charged with anything, but because the report was automated, there wasn't even anyone to sue for a false report.

[miohtama]

There is no such thing as not to sue anyone. Police can squeeze and lie as much as they want, but there are laws about the abuse of power, false police reporting, obstruction of justice. But it will be expensive as effectively you are going to the court against a state.

Also of course there is a person somewhere behind a keyboard who wrote the software which flags, correctly or incorrectly, files. Their name (Thorn) is kept strictly away from any public testimonial with NDAs with police, because eventually there will be class action lawsuits against them in the USA.

[lcnPylGDnU4H9OF]

> because the report was automated, there wasn't even anyone to sue for a false report

Is there a reason the legal entity which deployed the software can't be named? Seems like the next logical step, anyway.

[franga2000]

The thing is, it was technically a correct report. One frame of that gif did correpond to a known piece of CSAM (presumably they use some kind of perceptual hashing). The facts that 1) the gif was clearly a sick joke (he described it as a slow motion bullet shot from a movie, landing in something/someone and then flashing the one frame, presumably the intention being "haha, get shot with the child porn bullet"); 2) it was only one inconsequential piece, not a whole collection; 3) it was downloaded automatically from a group chat... are not in scope of the "did this user just upload CSAM to our servers" function (from what I understood, it was triggered by the picture being backed up to Google Photos or Apple's equivalent).

These are all things that, in a functioning system, the police officer receiving the report would take into account. If it's a first report, diaregard. If it's a second, check the file name that was also presumably in the report, see it's a Whatsapp folder and disregard it. If it's a third report or there are multiple pieces, get a warrant to run a CSAM scan on the person's device, go to their apartment, run it, see there's nothing else, close the case. If it's a clear "prank", start investigating the person who sent it.

But since the police are, in general, trigger happy lunatics, you get a full raid instead. And since computer forensics is hard and doesn't pay well, the investigation took many months instead of an afternoon. The fuckup was squarely on the law enforcement side, as well as in the law itself.

[amy_petrik]

>(he described it as a slow motion bullet shot from a movie, landing in something/someone and then flashing the one frame, presumably the intention being "haha, get shot with the child porn bullet")

That's the slippery slope nature of these laws. For sure a CSAM is "out there" and easily acquired. And now it some sort of toxic, radioactive content that destroys systems, corporations, and most importantly, invididuals if weaponized.

I suppose these people with good intentions, seeking to wipe CSAM off the face of the earth with religious fervor ... I suppose they never realized that such thing as a troll exists on the internet who will gladly point their fervor as the troll pleases like a firehose of seething

[aftbit]

This is one good reason we should not tolerate our devices auto-snitching on us to the police. Any tool can be weaponized. The legal system has a presumption of innocence, but it grinds painfully slowly, and the mere investigation can be extremely disruptive, even assuming they don't find anything further to pursue once they turn the eye of Sauron upon you.