[nickff]

But this statement only applies to a 'naive' (or first) statistical analysis or test on the dataset. Once the researcher starts changing their assumptions in response to the results they're seeing, they're p-hacking and p-values are no longer meaningful. In addition, once you have multiple researchers looking at the same dataset with different assumptions, and you factor in publication bias, the p-value also loses meaning.

[comte7092]

Well, yes and no. The p-value still means the same thing, but when you take a dataset and go looking for any result that is under a certain threshold, you’ll probably find it. “Unlikely” events happen all the time!

What your comment is highlighting is an issue with bad experimental design. (And, obviously, with our publication regime)