[radford-neal]

Umm... That's not actually what only publishing a result when the p-value is less than 0.05 accomplishes. Such a publishing policy instead guarantees (or would if there were no other problems) that __only 5% of the FALSE ideas that researchers test get published as being "confirmed"__ (i.e., when the null hypothesis is true, only 5% of the time does it get "rejected" at the 0.05 level). It guarantees nothing about what fraction of these supposedly-confirmed ideas are actually true. That fraction could vary from zero (if researchers never think of true ideas) to 100% (if researchers never think of false ideas).

[prestonh]

You're right, I was trying to dumb down p-values to make a point about perverse incentives in academic publishing. What's worse is that other factors that increase the Type I error rate (only positive results are published) only compound the effect.