[pacbard]

I saw a chart once that made the point that most research studies can be classified along two axes: rigorousness of methods and popularity of results. Most published papers have either high rigor/low popularity and low rigor/high popularity. The trade-off is in the fact that highly rigorous studies only allow for narrow, unexciting results, while popular studies with flashy results will have to compromise on their rigorousness. This is not really a rule, but it is an interesting way to see research and the editorial/peer-review process.

The example discussed in OP seems to fall in the category of low rigor/high popularity. I am not 100% on my history of psych research, but it seems to me that the stereotype threat was all the rage in the late 90s following the publication of Steele and Aronson (1995). OP study seems to follow a similar experimental setup as S&A with a new group of people (Asian-American women).

As far as meta-science is concerned, I think that it remains mostly a part of philosophy (as in epistemology) and the focus of a few (senior?) scholars in each field. There is really no space to publish meta-scientific papers that "shake up" the field and call out established researchers, as editors that publish those pieces could come under similar criticism for their work. I think that it is not an accident that the discussion of the replication crisis in psychology started from blog posts and other non-academic avenues and then found its way to more "established" publications in the field (again, if I remember the context of those conversations).

I really wish that the review process was open. It would be interesting to see the reviewers comments to this specific paper and how the editor decided to pick up and engage with them. All those conversations are usually locked up in some editorial management system and are seldom made public. I don't know if we can really have open science without having open peer reviews.

[derbOac]

I agree with you mostly, but it's worth noting that modern meta-science had its blossoming in psychology in the 1960s, with the development of meta-analysis (with educational psychology and clinical psychology). Technically the origins are much earlier, in the 30s(?) in statistics, but as a field I think it took off around that time, and spread.

Similarly, the replication crisis was being discussed in a lot of areas, especially in psychology, throughout this time, but was largely ignored until after the Bem ESP study. Registered replications aren't new, nor is concern about meta-science; it's just had renewed focus in recent years for various reasons.

It's not all that surprising to me that meta-science is associated with psychology. After all, not only is psychology often sort of fuzzy (by necessity of its subject matter), but it's the science of human behavior, which I think can lay claim to scientist behavior as well.

I think it's arguably the greatest contribution of psychology to the sciences in general.