[factorymoo]

As someone who has worked at big social media companies, I can say the conventional wisdom about content moderation being a carefully planned process is off-base.

The reality is that moderation relies heavily on imperfect machine learning models and overworked human reviewers making rushed judgments on hundreds of cases per day. There's no meticulous strategy document mapping out the pros and cons before banning accounts that upset the company.

Mistakes inevitably happen when relying on this combination of flawed automation and human reviewers who are stretched too thin. The moderation policies may seem arbitrary or politically motivated from the outside, but much of it comes down to hasty human error and buggy algorithms rather than some malicious scheme.