[wpietri]

My NDAs constrain me from saying as much as much as I'd like, but it's a mistake to think that the key to fighting abuse is "algorithms". The heart of it is always human judgment. It starts at the executive level to set policy. That policy needs to be carefully socialized to users. The user need ways to report problems. And then you need trained staff to judge the reported content.

Algorithms can help, of course. But the problem is mainly a human one.

[cudgy]

Humans are incredibly biased in making judgements like this. How do you “train” a person to enforce a standard that goes against their inherent biases?

Seems like the more objective the process can be shifted (by moving away from human decisions), the more effective/fair the process becomes.

Anyway, it sounds like you may work in this space, and, therefore, might have further insight, which I am very interested in hearing about.

It is definitely an Achilles Heel for social media platforms.

[wpietri]

Unfortunately, there is no real objectivity here. Machine learning systems are fed large numbers of human judgments, are tuned based on human judgment, and then are deployed when other humans think them ready.

The way you get reasonable consistency, whether it's humans or machines, is by establishing clear standards, using them for training, and then continuously monitoring results. It's not perfect, of course. But nothing is.