| I worked at Huffpost through all three of these phases in a technical capacity adjacent to comments and moderation, as director of technical operations and eventually as head of engineering. This study has significant questions to answer about their methods and assumptions that is summed up here: "Second, we know that HuffPo used both manual and algorithmic moderation in all three phases, but we do not know how the policies changed under the different identificatory regimes." Given whay I know about Huffpo's moderation system and their statement that they don't have understanding of them I'd say that nothing reported in this study should be considered valid for a few reasons. One is that Huffpost used many different systems of moderations following many changing standards throughout their days as a big news site (#3 in the US at one point) and the biggest news-based community, which they were for several years. They started with no moderation, then human moderation with evolving standards and practices that were overseen by a brilliant community team. Then they bought Julia in 2010(? ish) which was a very early machine learning moderation system that was trained on millions of human made moderation decisions before launch and whose training and internals were constantly updated and improved for years. Julia was dropped for Facebook comments later on, at which point Facebook did most of the basica moderation but was still assited by human moderators. My first critique of this analysis is that the authors have no data or understanding of the moderation actions that resulted in suppression of comments or users. How can an analysis make any claims without this data? So far as they know the comment-flow was more hostile during the periods they observe as more civil, but there were just far more comments suppressed. Just one of dozens of other internal details of the operation that would invalidate their conclusions would be that Huffpost quiet-deleted comments for a significant period of time -- meaning that you could post, and you would see your posts in context when you were logged in but no one else would see them. They also silent-banned users. This and other details of implementation create a great deal of complexity and secondary effects. I can attest that moderation was very, very active and that lots and lots of comments were moderated down and out of the comment threads... again indicating significantly less civility than any retrospective analysis would be able to discern without all the data. I also find it interesting that this study chose Huffpost for the analysis. At the site's hights of success and profit the comment threads were the reason for their SEO dominance and were considered to be the most important secret sauce. Huffpost moderation was the best in the business by a long measure. With the methodology presented it would make sense to me to say that Huffpost would appear to be the most civil of the big sites of the time. So it is interesting that this study focuses singly on Huffpost and reports that their theories indicate this differential. While the authors do cover some of this in their section on Limitations, they don't cover near enough to justify their results... instead this reads as another cherry-picking study where authors had a theory and found a dataset that confirmed it while being unaware of fundamental reasons why that dataset was an outlier, making it impossible for them to build in the needed controls in their methods. |
At first they had their own accounts to sign up for on the main website, there were definitely some unsavory characters and trolling but I'd say by and large it was just normal commenting. They announced that due to abuse or moderation issues (I can't recall which) they were switching to facebook commenting, which ostensibly has a real names policy.
A month later comments were removed from the website altogether. The only users left were some of the nastiest posters ever and didn't seem concerned about their real name being up there next to the consistently awful things they had to say, possibly because they were mentally ill. I know I had no desire to interact with them and using my real name on a site full of crazy people sounded like only something a crazy person would do.