[Estragon]

> a p-value is the probability of getting your experimental results given that your hypothesis is wrong.

That's a common misconception. Actually it's the probability of getting your experimental results given that your null hypothesis is right.

[vlehto]

So "probability to perceive what you are perceiving just because of dumb luck"?

Then research with high p-value would just mean "we tried, and we still know practically nothing".

[im2w1l]

Well normally, hypothesis is wrong <-> null hypothesis is right.

[iamsohungry]

No, if the null hypothesis is right that implies that the hypothesis is wrong, but the implication relationship does not go the other direction.

Consider an experiment where your hypothesis is that cold temperatures cause the common cold. This is a good example for a thought experiment because we "know the answer" in a way (there have been a lot of experiments on this). The null hypothesis in this case is that cold temperatures are uncorrelated with incidence of the common cold.

You place people in isolation in cold areas and a control group in warm areas, and study how many get the common cold. None of the people who didn't already have colds get the common cold: because they are in isolation and the common cold is caused by rhinoviruses (which they can't get because they are in isolation), you get exactly the same results.

This disproves the hypothesis, but it does not prove the null hypothesis, that cold temperatures and the common cold are unrelated. Cold temperatures are, in fact, related to the common cold.

Try a second experiment: you place people in groups of five in cold areas and in warm areas, and discover a moderately high correlation between cold temperature and incidence of the common cold. This disproves the null hypothesis. But the simple hypothesis that cold temperature causes the common cold has also been disproven by your first experiment.

The reason for this is that the correlation between cold temperature and the common cold is a dependent correlation: given that rhinovirus is present in the system cold temperature is correlated with incidence of common cold (rhinoviruses reproduce ideally at temperatures significantly lower than human homeostatic temperature).

The null hypothesis is not just a statement that there is no independent correlation, it's a statement that there is no independent or dependent correlation. As such, the null hypothesis is an extremely broad hypothesis which is impossible to practically prove. This is why there's such a focus on finding correlations rather than finding non-correlations: you aren't going to prove the null hypothesis.