[cygaril]

A more general method is to use the likelihood ratio, ie the ratio of the likelihood of an outcome under the alternative hypothesis to its likelihood under the null hypothesis. And then pick the outcomes which for which this ratio is highest as the ones which will cause you to reject the null hypothesis. Equivalently, the p-value is the probability under the null hypothesis that the likelihood ratio would be at least this large.

This works in the discrete case too, and gives p=1/32 in the original coin flip case.

[6gvONxR4sf7o]

Is the likelihood ratio test more general? I thought that one of the benefits of the usual NHST framework was that you only need the distribution of your stat under the null. With LRT don't you need the distribution under both the null and the alternative? How do you frame a null of mu = 0 against an alternative of mu != 0 with x ~ D_mu in this way?

[cygaril]

You don't necessarily need the distribution under the alternative to determine the values for which the likelihood ratio will be highest. In your example, the tails will be the areas of maximum likelihood for any (symmetric) alternative.

[6gvONxR4sf7o]

Huh, TIL. Thanks :)