[prestonh]

What's good for the field (we've decided) is when we only allow a 5% Type I error rate (95% of published positive results are correct). What's good for the individual is to drive their Type I error rate up just enough so that they can publish as much as possible without getting caught.

[radford-neal]

Umm... That's not actually what only publishing a result when the p-value is less than 0.05 accomplishes. Such a publishing policy instead guarantees (or would if there were no other problems) that __only 5% of the FALSE ideas that researchers test get published as being "confirmed"__ (i.e., when the null hypothesis is true, only 5% of the time does it get "rejected" at the 0.05 level). It guarantees nothing about what fraction of these supposedly-confirmed ideas are actually true. That fraction could vary from zero (if researchers never think of true ideas) to 100% (if researchers never think of false ideas).

[prestonh]

You're right, I was trying to dumb down p-values to make a point about perverse incentives in academic publishing. What's worse is that other factors that increase the Type I error rate (only positive results are published) only compound the effect.

[Will_Parker]

A 0.05 p value very much does not mean there's a 95% likelihood of the result being true. It's a fallacy that is far too common.