[logankeenan]

Regardless of how you feel about content moderation, 48 hours is a ridiculously long time given what AI can do today. That “bad” image could have been propagated around the world to millions of people in that time. It can and should be removed in minutes because AI can evaluate the “bad” image quickly and a human moderator isn’t required anymore. However, the compute costs would eat into profits…

Again, I’m not judging about content moderation, but this is an extremely weak initiative.

[Ukv]

> It can and should be removed in minutes because AI can evaluate the “bad” image quickly and a human moderator isn’t required anymore.

CSAM can be detected through hashes or a machine-learning image classifier (with some false positives), whereas whether an image was shared nonconsensually seems like it'd often require context that is not in the image itself, possibly contacting the parties involved.

[voidUpdate]

I would not want to be the supervisor that has to review any CSAM positives to check for false ones

[thaumasiotes]

> CSAM can be detected through hashes or a machine-learning image classifier (with some false positives), whereas

Everything can be detected "with some false positives". If you're happy with "with some false positives", why would you need any context?

[pjc50]

Indeed. It seems that the process being described is some kind of one-stop portal, operated by or for OFCOM or the police, where someone can attest "this is a nonconsensual intimate image of me" (hopefully in some legally binding way!), triggering a cross-system takedown. Not all that dissimilar to DMCA.

[Manuel_D]

The issue is that if you need to achieve 0% false negatives, you're going to get a lot of false positives.

[timr]

Another, related issue is that the takedown mechanism becomes a de facto censorship mechanism, as anyone who has dealt with DMCA takedowns and automated detectors can tell you.

Someone reports something for Special Pleading X, and you (the operator) have to ~instantly take down the thing, by law. There is never an equally efficient mechanism to push back against abuses -- there can't be, because it exposes the operator to legal risk in doing so. So you effectively have a one-sided mechanism for removal of unwanted content.

Maybe this is fine for "revenge porn", but even ignoring the slippery slope argument (which is real -- we already have these kinds of rules for copyrighted content!) it's not so easy to cleanly define "revenge porn".

[bluGill]

DMCA isn't directly that bad. DMCA is under penalty of perjury, so false take downs are rare.

The problem is most take downs are not actually DMCA, they are some other non-legal process that isn't under any legal penalty. Though if it ever happens to you I suspect you have a good case against whoever did this - but the lawyer costs will far exceed your total gain. (as in spend $30 million or more to collect $100). Either we need enough people affected by a false non-DMCA take down that a class action can work (you get $0.50 but at least they pay something), or we need legal reform so that all take downs against a third party are ???

[timr]

> DMCA is under penalty of perjury, so false take downs are rare.

Maybe true with the platonic ideal "DMCA takedown letter" (though these are rarely litigated, so who really knows), but as you note, they're incredibly common with things like the automated systems that scan for music in videos (and which actually are related to DMCA takedowns), "bad words" and the like.

> The problem is most take downs are not actually DMCA, they are some other non-legal process that isn't under any legal penalty.

It's true that most takedowns in the US aren't under DMCA, but even that once-limited process has metastasized into large, fully automated content scanning systems that proactively take down huge amounts of content without much recourse. Companies do this to avoid liability as part of safe harbor laws, or just to curry favor with powerful interests.

We're talking about US laws here, but in general, these kinds of instant-takedown laws become huge loopholes in whatever free speech provisions a country might have. The asymmetric exercise of rights essentially guarantees abuse.

[worthless-trash]

I believe google issues legitimate dmca takedowns for copyright strikes, even when there is no infringement. They put the work to defend the strike on the apparent commiter, often with little to no detail.

While the false takedown may be rare. Using dmca as a mechanism to inflict pain where no copyright infringement has taken place is indeed common enough that it happens to small time youtubers like myself and others I have talked to.

[Ray20]

Regardless of how you feel about content moderation, we are talking about a situation where the government is DEMANDING corporations to implement automated, totalitarian surveillance tools. This is the key factor here.

The next step would be for the government to demand direct access to these tools. Then the government would be able to carry out holocausts against any ethnic group, only 10 times more effectively and inevitably than Hitler did.

[happymellon]

> the government is DEMANDING corporations to implement automated, totalitarian surveillance tools

You've got this the wrong way around. These are social media sites.

People are publicly publishing revenge porn, and the government has told sites that if they are requested to take down revenge porn then they have to.

They don't have to monitor, because they are being told of it's existence.

[oneeyedpigeon]

Are you conflating this specific principle with the much wider Online Safety Act? Because, while the latter has certain privacy-undermining elements to it, I'm not sure how asking social media companies to take down content has anything to do with 'surveillance'.

[Ray20]

No, I'm not conflating. The point here is that for taking content down in the way they want it, the implementation of totalitarian surveillance tools is necessary.

[Manuel_D]

As bad as I think this law is, this isn't demanding any degree of surveillance in the sense that real human beings have their information or activity tracked. This is mandating taking down content, not surveiling anyone.

[Ray20]

> This is mandating taking down content, not surveiling anyone.

As far as I understand, it precisely mandating to monitor EVERYONE.

They are not talking about removing a specific image from the platform based on its hash or something. They are talking about actions that involve automated analysis of all content on the platform for patterns arbitrarily specified by the government.

The technologies discussed differ from totalitarian surveillance by simply toggling a single flag on the platform, and are indistinguishable from such surveillance for the user.

[roryirvine]

Every social media site is already reading, analysing, and filtering the content posted on their platforms.

[Manuel_D]

Imagine dang wrote a script to delete every HN comment that contains the string "velociraptor". Under your logic, this involves surveiling every HN commenter. This is true in the pedantic sense that every comment posted to the site would be checked for "velociraptor".

But most people understand the word "surveillance" to mean more involved information collection than just deleting content that matches certain criteria.

[pjc50]

Every social media site already has a system for removing porn.

[Ray20]

These systems does not imply the tools of totalitarian surveillance.

In contrast to the proposed one, which should be able to classify all content on the platform at any arbitrary moment in time according to a post-factum specified arbitrary filter. Literally a mechanism of totalitarian surveillance.