[unishark]

P-values are based on a normally-distributed sampling distribution which presumes the samples are randomly-chosen. It's hard to see how random sampling can apply very well here.

Suppose we did a comparison between the p-values of biased researchers desperate to publish, versus a conservative heuristic that doesn't believe small effects. Particularly for social science experiments like this, which would you bet on being able to assess repeatability better?