Hacker News new | ask | show | jobs
by euroclydon 4452 days ago
> even if I have a coin biased badly against coming up heads, if I toss it enough number of times I will get `x` heads.

I'm pretty sure a "trial" is a statistically significant study of many patients. So in this case, your analogy to the coin is flawed. A better analogy would be x groups of 10,000 coin flips, and to take the average of each group in x. Now, no such average would favor the less probably side of the coin.

1 comments

No, grouping over many patients does not eliminate the loop hole at all. The 'coin' was my analogy for the event that a trial was deemed successful (according to some statistical test of hypothesis). If all that I need is to demonstrate is 2 trials that show statistically significant benefits, I can do that expenses permitting (unless the probability of benefit using my candidate drug is exactly 0). The key is that I am allowed unlimited trials where my candidate may not only show no statistically significant benefit but also show worse performance. If only the decision to accept my candidate is based on all the trials together is this hole plugged. My understanding is that FDA does not do this. 2 strikes and you are in, regardless of failure in the other trials.