|
|
|
|
|
|
by sjeohp
2774 days ago
|
|
|
The assumptions are: 1) third party fact-checkers (eg. snopes) are credible and reliable. 2) the third party "Botometer" (machine-learning model) is valid and was trained on an accurately classified corpus. This type of study always relies on the elevation of someone's subjective assessment (eg. "this account is a bot/human", "this fact-checker is honest") to the status of objective truth. It is a serious weakness when studying subjects loaded with political consequence, wherein well-resourced stakeholders are constantly attempting to shape competing narratives by any means available to them. |
|
|
>2) the third party "Botometer" (machine-learning model) is valid and was trained on an accurately classified corpus.
For anyone curious enough to follow up on how effective their "botometer" tool was, the methods section [0] details what they used and links to said tools. The language used is simple enough for most people to follow. I am not great at following their statistics, but the botometer seems to be at least decently accurate. Certainly, figure 2d seems pretty convincing to me for validating the botometer.
>1) third party fact-checkers (eg. snopes) are credible and reliable.
The methods section talks about how their model works with Onion articles. The great thing about The Onion is that it is a perfect positive control i.e it is always false. The authors state the following in the methods section:
>Many low-credibility sources label their content as satirical, and viral satire is sometimes mistaken for real news. For these reasons, satire sites are not excluded from the list of low-credibility sources. However, our findings are robust with respect to this choice. The Onion is the satirical source with the highest total volume of shares. We repeated our analyses of most viral articles (e.g., Fig. 3a) with articles from theonion.com excluded and the results were not affected.
0. https://www.nature.com/articles/s41467-018-06930-7#Sec8