Hacker News new | ask | show | jobs
by iamsb 1980 days ago
My suggestion is to not indulge in any content moderation which is not illegal. Only take down content which is required by a court order. Limit use of automated content moderation only for easy to solve cases like child pornography.

Why?

It is fairly clear at this point that content moderation at internet scale is not possible. Why? A. Using other users to flag dangerous content is not working. Which users do you trust to bestow this power with? How do remove this power from them? How do you control it becoming a digital lynch mob? Can you have users across political, gender, other dimensions All mostly not solvable problems.

B. Is it possible to use machine learning? To some extent. But any machine learning algorithm will have inherent bias, because test data will also be produced by biased individuals. Also people will eventually figure out how to get around those algorithms as well.

The causality between content published on the internet and action in real world is not immediate. It is not like someone is sitting in a crowded place and shouting fire causing a stampede. As there is a sufficient delay between speech and action, we can say that the medium the speech is published in is not the primary cause of the action, even if there is link. Chances of direct linkage are fairly rare and police/law should be able to deal with those.

Content moderation, at least the way Twitter has been trying to do, has not been effective, created lot of ways for mobs to enforce censorship, and there is absolutely no real word positive impact of this censorship is. Only use of this moderation and censorship has been for right to claim victimhood and gain more viewer/readership to be honest.

3 comments

You realize that the approach you suggest pushes out a different set of people, right?

For example, a soldier with PTSD may want an environment that moderates content. Or a journalist with epilepsy may want a platform where people don't spam her with gifs designed to trigger epilepsy when she says something critical of a game release.

I understand. Most of those can be achieved using privacy and sharing settings though and does not necessarily require content moderation.
Doesn't that require the active cooperation of bad actors? Sure, you can create a filter to hide all posts tagged with "epilepsy-trigger", but that doesn't help if the poster deliberately omits that tag. Allowing users to tag other people's posts patches this issue, but opens up the system for abuse by incorrect tagging. (E.g. Queer-friendly posters being flagged and demonetized after being maliciously tagged as "sexual content".)

At some point, there needs to be trusted moderation.

I’d point out that child sexual abuse content is not an “easy to solve case”. The extent to which Facebook, Google, and more recently, Zoom look the other way on this issue is horrifying and it seems to be a very hard problem due to the laws surrounding the handling of such material. I’m not faulting the laws, I just think this is an inherently hard issue to crack down on. Gabriel Dance and Michael Keller at the NYT did some very high quality reporting on this whole issue in 2019 (https://www.nytimes.com/interactive/2019/09/28/us/child-sex-...).
That link is pay/auth-walled for me, but I do get your point. Perhaps easy to solve was underestimate the technical problem as I was more thinking in terms of political problems. From that perspective no one, other than pedophiles themselves, disagrees with removing that kind of content. But completely agree on tech side of it.
So it would be OK for a kids TV show website to have Viagra ads on it?

Edit: I mean spam.

That would be a stupid waste of money for an advertiser. Maybe ad networks are really the problem.
I am only commenting on this in the context of community standards and user generated content and should not be extrapolated to all content in all other contexts.
Sorry, I meant spam. If anyone can post a comment, and only "illegal" comments can be removed, that would surely allow a lot of email-style spam.
That is a fair question/comment. In my original comment "Limit use of automated content moderation only for easy to solve cases like child pornography." . It is reasonable to extent that list beyond just child pornography. I did not intend to give impression than this list is exhaustive.

In case of spam - Emails do show you spam emails, just hide them behind a spam folder. So instead of outright removal, it is possible to use similar techniques. And let users decide whether they ever want to see spam comments.

> It is reasonable to extent that list beyond just child pornography.

Seems like you're just back to square one there.

You're making a list of unacceptable content. Whatever you put on there, someone's going to disagree.

Whether it's automated or not probably isn't the issue.

I see a lot of comment sections ruined by the typical bot spam (e.g. "I earn $5000 a minute working from home for Google").