[pradn]

What if hashes of known-bad content are stored locally on the device, and sending content that matches against those hashes is not allowed. Or, the user could appeal if they think there's a false positive. This can be used for CP but also for known-bad fake news or inflammatory content. Clearly, the content hash DB needs to be scope down, and what goes in there should be chosen with democratic principles, and stand scrutiny in the courts. If done thoughtfully, it seems like a feasible solution.

[ApolloFortyNine]

Changing a hash is incredibly easy, you could just change some Metadata and the hash would change. And any perceptual hashing algorithms would naturally lead to false positives.

Also this would likely be quickly commandeered for copyrighted work (honestly pretty surprised it hasn't happened already).

[ergl]

This is already done extensively: https://en.wikipedia.org/wiki/PhotoDNA and https://blog.cloudflare.com/the-csam-scanning-tool/

[pradn]

Yes, it would have to be a perceptual hash. False positives will occur, so there needs to be a way to appeal or remediate the algorithmic decision. We already apply this approach in a bunch of places. I believe the major personal cloud storage providers (OneDrive, etc) already do such scanning.

[ApolloFortyNine]

>This can be used for CP but also for known-bad fake news or inflammatory content.

It worries me that anyone thinks it would be a good idea to have "fake news" and "inflammatory content" blocked at the device level. Obviously cloud providers can do whatever they want (though I doubt it catches any more than the lowest hanging fruit, encrypting then uploading would be uncatchable), but the idea that my device will have a list of disapproved content, and I'll have to appeal to the government to be allowed to view it in case of false positives? The day that becomes a reality freedom will truly be dead.

[pradn]

I didn't say it would be at the system level. I'd expect this to happen per app. It's similar to how photo manipulation software can detect currency. I doubt every such app complies, and certainly the system screenshot tool does not.