[Sniffnoy]

...as best I can tell, you have still swapped them? It sounds like you're still talking about P(null hypothesis|evidence), whereas p-values are about P(evidence|null hypothesis). (Well, not quite the latter but something like it.)

[acover]

Yes. Ugh. You are right.

I treat P(evidence | null hypothesis) equivalent to P(null hypothesis | data). When it should be

P(evidence | null hypothesis) = P(null hypothesis | evidence) * P (evidence) / P (null hypothesis)

As we don't know either of those extra terms we can't determine P(null hypothesis | evidence).

Thanks.

Interestingly wikipedia notes that P(Reject H | H) = pre-defined threshold for rejecting H [1]

[1] - https://en.wikipedia.org/wiki/P-value