[novok]

It's a statistical inevitability that this will happen all the time at facebook's scale, even if you have an algorithm with a %0.00001 false positive rate, because they deal with billions of items per day. It's really actually fairly rare unless your purposefully bashing your content against their stated boundaries, but huge numbers make thousands of these events happen per day around the world.

Just like society has learned to accept certain rates of death, crime, medical imperfection and more to have a functioning society, we will have to come to terms that algorithmic moderation will also go wrong and we will have to pay for it, with $$$, one way or another. It will never be perfect, because what is perfect is different to different groups.

[smsm42]

I don't think an appropriate reaction to an instance of murder, robbery or medical malpractice in our society is "well, as a society, we learned to accept certain rates of death, crime, medical imperfection, so move along, nothing to talk about here". That doesn't look like a society I am familiar with.

[novok]

It is the society you are familiar with, it's just everyone treats it as normal and what are the limits of possibility so you are probably blind to it.

Some relatively extreme examples:

1. There isn't a cop / security person in every corner, to really prevent murder and robberies. After a certain point, society decided it wasn't worth the cost to prevent murder in this expensive way. You should dig into what the murder / robbery solving rate is in your own home, you might be surprised!

2. Cars kill a lot of people, all the time. We would save lives if the speed limit was reduced to 10kmh. But that makes cars not very useful. Our cars are not volvo tanks and miss a lot of safety testing that volvo does and goes above and beyond, partly because of cost.

3. We don't put highway style concrete barriers around every road boundary to prevent human death.

4. The success rate of medicine to save lives and prevent disability is accepted to be not perfect and socialized medical systems do not spend unlimited money to save X amount more of people.

5. Obesity is probably the largest killer and life reducer of humanity in the USA today, yet there isn't anywhere near a proportional response by US society to fix this like there was with COVID, smoking or 9/11.

6. I don't know if your american, but the entire insulin price jacking controversy kills people constantly today in america. That society has basically said the rich pharma companies rather make more money and kill people who cannot afford this insulin vs. save lives.

All of these things are way worse the 5 9s of reliability that are algorithmic moderation systems, but we accept the tradeoff because sometimes, the cost is worst than the cure. And sometimes, for way worse and gross reasons, like the american insulin example

[smsm42]

Sometimes it is. But I do not see any reason why auto-moderation system that does not ban "Merry Christmas" should cost immensely more than one that does. I can see why 10mph speed limit would be very costly, but why not having a broken model is more costly than having a broken one? Where the costs are coming from? I don't think the costs are the problem here. They spent a lot of effort on attaching various warnings to every post mentioning covid, vaccines, etc. They could spend a little on making their models not to suck. They didn't. I don't think "it would cost too much" is a valid excuse - at least without demonstrating why it'd cost too much.