Hacker News new | ask | show | jobs
by kristjansson 1693 days ago
The argument against p-values is part of the argument against any bright-line single-number rule for identifying truth. The job of the researcher is to demonstrate (at least) that

1. there is (isn't) an observable difference between groups of interest

2. the difference is (not) attributable to hypothesized causal mechanism i.e. the (absence of a) difference isn't due to random variation in the observed sample i.e. the difference would be observed by a independent replication of the same analysis/study

3a. the difference is not explainable by other factors that vary between the groups, observed or unobserved

3b. the difference is not artificially inflated (suppressed) by the statistical choices

4. the difference is large enough to be practically relevant.

and so on

If the degree of certainty of statements about the difference can be characterized by a single number at the end of the process, great! But the goal should be a convincing, wholistic story, not the single number.