[thaumasiotes]

Andrew Gelman frequently talks about a problem in the scientific literature, in which you see the following pattern:

1. A study is done at the typical (very low) level of quality, and it gets published somewhere.

2. A more careful study tries to replicate the effect and finds that it doesn't exist.

3. The world concludes that the effect doesn't occur in the very specific context observed in the second study, but does occur in all other contexts.

Gelman makes the point that if the two studies occurred in the other order -- first, a high-quality study finds no effect, and second, a low-quality study finds an effect -- no one would conclude the effect was generally present. But the order in which you perform studies is not actually relevant to anything.