[realradicalwash]

so a categorical variable got mixed up as a numerical one and produced misleading results.

to the credit of the authors, they released their data sets. -- however, i suspect that proper data exploration and visualisation would have prevented all this. visual inspection would have most likely revealed that there is no visible effect, or even an effect in the opposite direction, and once you see this, all alarm bells should go off if your model predicts otherwise. so i suspect that the authors skipped some basic steps and got carried away by results that promised a nice headline.

[rossdavidh]

Agreed. However, another factor may be that, we tend to scrutinize more closely results that don't line up with what we expected. In fact, in this case it was another researcher who had gotten results that pointed in the opposite direction, who convinced the original researcher to release their data (which, to their credit, they did).

Which, is one reason why having higher and higher percentages of academia and science researchers be from the same part of the political spectrum, is worrisome to me. If you have more diversity in ideology, there is more likely to be someone in each field to have the instinct to scrutinize closely a result which, when scrutinized, won't hold up.