[dejb]

> In reality, if you publish 20 experiments with p=0.05 [1], 1 of them should be incorrect.

In reality it doesn't turn out this way because the results that get written and published tend to be biased in favour of novelty and demonstrating a relationship rather than the absence of one. How many similar experiments could have been terminated, never submitted or not published because they failed to show anything notable? This is one of reasons... 'Why Most Published Research Findings Are False' http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/

That meta-study applied to medical studies and I think this genre would probably fair even worse when it came to long term replicability.

[sesqu]

And to go the other way, it doen't happen like that because the rule of thumb isn't p=0.05, it's p<=0.05 - and p can be quite small indeed, if you run out of ideas before running out of data (such as might happen in a novel area).