| Certain groups have been serious the replication crisis for 10-15y, but academic culture at large is simply not cut out to discuss fraud in a 'street epistemology' sort of way, such as the way security researchers might discuss cybercrime. There's a wild amount of pushback to any amount of meta-criticism. But once you get past that point, many roadblocks remain. In particular, there's extreme bias for meta-statistical methodologies that infer QRPs over other methods of investigation. Often, these methods aren't strictly necessary in context, and afford the opportunity to turn the metasci discourse into endless bikeshedding about the meta-framework rather than the object of dispute. Many interested parties will participate/perform in this discourse, but few will sit down and really look at things as simple as the logical structure of the paper's claims, or even simpler problems with its content. In the case of social psych, for instance, many problems lie with (a) stimuli and (b) unexamined assumptions on the part of the researchers that a certain manipulation holds, so these are important to assess. But extending critique to these things is seen as "reviewer 2" behavior — uncollegial, unfair, sniping, waaah etc. Since "researcher x produces bad research, but evidence of fraud is only circumstantial" isn't sufficient grounds for doing much of anything, little comes of these efforts. When an investigation does nail a fraudster to the wall, well, so? The papers remain, the poisoned citation tree remains, the culture remains. More than anything, the pushback in the form of tone-policing is what gets me. "Methodological terrorism" this, "reviewer 2" that. It shows how far from consequences the gatekeepers are. If you're a young person, whole branches of the academy have been pre-bankrupted for you. There's no hope there, short of a research path that manages to avoid citing any prior literature. But don't get tetchy with the grantlords! Meanwhile, all around the US, real effects of this dogshit excuse for scientific inquiry can be seen every single day. Police departments are adopting new policies around known-bad implicit bias papers. These won't work. We know they won't work. We've known this for years. What would make it stop? Every time cops kill an unarmed person, academics who still haven't retracted their implicit bias papers get fined? There's the rub. You can't do much to force the point. Effective, timely measures would be fairly brutal ones, and academics aren't ready to admit this to themselves. In some ways, COVID-19 may end up being one of those measures, though it will disproportionately affect younger researchers. |