[ggrothendieck]

Not being statistically significant is not a proof that it doesn't work -- it only means they could not reject the possibility that the results were due to chance. The possibility that it slashed the death rate by 3x (which is what happened in the study) when projected to the world wide deaths of ~ 4.5 million would imply saving the lives of 3 million people so it certainly would be worthwhile to check it out. Maybe it was due to chance but maybe it was not.

[awb]

> The possibility that it slashed the death rate by 3x (which is what happened in the study) when projected to the world wide deaths of ~ 4.5 million would imply saving the lives of 3 million people so it certainly would be worthwhile to check it out. Maybe it was due to chance but maybe it was not.

When you talk about 3x, you're talking about 10 vs. 3. Extrapolating that out to millions of people is not a great idea.

Let's say you run an ice cream company. You round up 490 friends and ask them to pick their favorite flavor of ice cream: chocolate or vanilla. 477 say they don't eat ice cream, 10 pick chocolate and 3 pick vanilla. You rework your ice cream production to be 3x chocolate : 1x vanilla based on your survey and promptly go out of business.

That's what's going on here as well, there's just not enough statistical significance between the two outcomes to infer any reduction in severe COVID cases.

> Findings: In this open-label randomized clinical trial of high-risk patients with COVID-19 in Malaysia, a 5-day course of oral ivermectin administered during the first week of illness did not reduce the risk of developing severe disease compared with standard of care alone.

[ggrothendieck]

Also they say they used the Fisher exact test and got a p value for mortality of 0.09 so it seems they were doing a two-sided test which is the default for fisher.test in R.

  ivm <- c(3, 247-3)
  con <- c(10, 249-10)
  m <- rbind(ivm, con)

  fisher.test(m)$p.value
  ## [1] 0.08809225

However, I think a one sided test could be justified and in that case it is significant at the 5% level.

  fisher.test(m, alternative = "less")$p.value
  ##        ivm 
  ## 0.04541928

[ggrothendieck]

Furthermore a test just twice as large would be sufficient to determine significance at the 1% level even with a two sided test if the same death rate continued to hold.

  ivm <- c(3, 247-3)
  con <- c(10, 249-10)
  m <- rbind(ivm, con)
  fisher.test(2*m)$p.value  # 2* so that it is twice as large
  ## [1] 0.008490957

[s1artibartfast]

Out of curiosity, which fisher test was used for the other endpoints?

[ggrothendieck]

The Mech Vent and ICU also used two sided tests.