[slg]

>Goalposts will shift. We will, hmm, go from suspending accounts which promote the theory that COVID-19 is caused by 5G towers, to suspending the account of a virologist who issued a preprint suggesting that gain-of-function mutations in SARS2 point to a laboratory origin.

Why do you believe that only the the second viewpoint will suffer from this slippery slope? We also seen similar examples in the first worldview. A conversation can quickly go from being against illegal immigrants, to being against all immigrants, to being against a specific race of immigrants, to genocide of that race. We have already seen this laissez-faire approach from Facebook help contribute to genocide in Myanmar.

[edmundsauto]

In fairness, Facebook's culpability in the Myanmar incident is because they launched in a language when they didn't have any moderators or AI who could understand the language. Further, much of the info was spread via memes - while extracting text from images is pretty good now, it wasn't always this way.

[punkrex]

They were repeatedly warned by local groups, international NGOs and the US State Department that the speech was inciting violence. They refused to do anything and it escalated into a pogrom.

[edmundsauto]

Not defending FB here - it looks like they have acknowledged their faults in this issue specifically.

That said, this argument structure sounds a lot like "US leadership was warned about the attacks on Pearl Harbor". It looks like FB under reacted to these warnings, probably because they didn't realize how bad the outcome would be. How can info/escalations be presented so as to break out of the noise? (I'm assuming here that FB also has been warned of a lot of really bad things that never came to pass, which isn't something we can know - but it's an interesting thought experiment.)

What is the expectation in terms of separate the signal from the noise? How can the critical factors be identified ahead of time? Was it foreseeable that the targeted hate speech would turn into violence? What level of reaction is appropriate, given the uncertainty of hate speech -> violence?

Apologies for the brain dump - not expecting answers to all of them. And not defending FB here. I just think these types of questions are very interesting (plus I just read Superforecasters, which examines similar decision making w/r/t the decision to kill bin Laden).