[PragmaticPulp]

Once you start dealing with small numbers (e.g. 2% versus 3%) then you would need far, far more patients to reach statistical significance.

It's tempting to look at things like 8 people visiting the ICU in one group but only 6 people in the other group and see that 6 < 8, but the problem is that it's too small of a sample size to decide if it's significant. The article covers that:

> There were no significant differences between ivermectin and control groups for all the prespecified secondary outcomes

The only one that almost comes close is death rate:

> The 28-day in-hospital mortality rate was similar for the ivermectin and control groups (3 [1.2%] vs 10 [4.0%]; RR, 0.31; 95% CI, 0.09 to 1.11; P = .09)

If this was the only Ivermectin study out there, it would be worth following up on. But it's not, and when this is added to the rest of the (not-retracted) studies it doesn't really change the picture.

At this point it matters less and less anyway. Countries that already tried Ivermectin at scale are starting to abandon the approach. Legitimately effective COVID drugs like Paxlovid with highly significant differences are becoming readily available. It's time to stop grasping at straws and accept that it doesn't work.

[s1artibartfast]

You are right that the study simply isn't powered to detect a decrease in mortality, even if it is there! That said, if true, a 70% reduction in mortality would still be of significant benefit.

You are right that this study doesn't change the picture. It is is just another underpowered study showing a large but statistically insignificant reduction in mortality. Yes, Paxlovid is almost certainly better.

That said, I would like to understand the efficacy of ivermectin with an appropriately powered and designed study. I hope ACTIV-6 reports out this year and used a reasonable treatment dose and timing comparable to Paxlovid.

[ggrothendieck]

Not being statistically significant is not a proof that it doesn't work -- it only means they could not reject the possibility that the results were due to chance. The possibility that it slashed the death rate by 3x (which is what happened in the study) when projected to the world wide deaths of ~ 4.5 million would imply saving the lives of 3 million people so it certainly would be worthwhile to check it out. Maybe it was due to chance but maybe it was not.

[awb]

> The possibility that it slashed the death rate by 3x (which is what happened in the study) when projected to the world wide deaths of ~ 4.5 million would imply saving the lives of 3 million people so it certainly would be worthwhile to check it out. Maybe it was due to chance but maybe it was not.

When you talk about 3x, you're talking about 10 vs. 3. Extrapolating that out to millions of people is not a great idea.

Let's say you run an ice cream company. You round up 490 friends and ask them to pick their favorite flavor of ice cream: chocolate or vanilla. 477 say they don't eat ice cream, 10 pick chocolate and 3 pick vanilla. You rework your ice cream production to be 3x chocolate : 1x vanilla based on your survey and promptly go out of business.

That's what's going on here as well, there's just not enough statistical significance between the two outcomes to infer any reduction in severe COVID cases.

> Findings: In this open-label randomized clinical trial of high-risk patients with COVID-19 in Malaysia, a 5-day course of oral ivermectin administered during the first week of illness did not reduce the risk of developing severe disease compared with standard of care alone.

[ggrothendieck]

Also they say they used the Fisher exact test and got a p value for mortality of 0.09 so it seems they were doing a two-sided test which is the default for fisher.test in R.

  ivm <- c(3, 247-3)
  con <- c(10, 249-10)
  m <- rbind(ivm, con)

  fisher.test(m)$p.value
  ## [1] 0.08809225

However, I think a one sided test could be justified and in that case it is significant at the 5% level.

  fisher.test(m, alternative = "less")$p.value
  ##        ivm 
  ## 0.04541928

[ggrothendieck]

Furthermore a test just twice as large would be sufficient to determine significance at the 1% level even with a two sided test if the same death rate continued to hold.

  ivm <- c(3, 247-3)
  con <- c(10, 249-10)
  m <- rbind(ivm, con)
  fisher.test(2*m)$p.value  # 2* so that it is twice as large
  ## [1] 0.008490957

[s1artibartfast]

Out of curiosity, which fisher test was used for the other endpoints?

[ggrothendieck]

The Mech Vent and ICU also used two sided tests.

[msandford]

So on the one hand I completely agree with you on the necessity of having enough people to dodge the problem of random chance. "the law of large numbers" on Wikipedia is good.

On the other hand you have three different categories where the numbers from one group are smaller than the numbers from the other group. Could it all be random chance? Sure! But that does kind of hint that there might be something there.

[awb]

> But that does kind of hint that there might be something there.

The paper does not draw this conclusion. The data you're referencing is too small to be statistically significant.

> Findings: In this open-label randomized clinical trial of high-risk patients with COVID-19 in Malaysia, a 5-day course of oral ivermectin administered during the first week of illness did not reduce the risk of developing severe disease compared with standard of care alone.

> Meaning: The study findings do not support the use of ivermectin for patients with COVID-19.

> Results: Among 490 patients included in the primary analysis (mean [SD] age, 62.5 [8.7] years; 267 women [54.5%]), 52 of 241 patients (21.6%) in the ivermectin group and 43 of 249 patients (17.3%) in the control group progressed to severe disease (relative risk [RR], 1.25; 95% CI, 0.87-1.80; P = .25). For all prespecified secondary outcomes, there were no significant differences between groups.

> Conclusions and Relevance: In this randomized clinical trial of high-risk patients with mild to moderate COVID-19, ivermectin treatment during early illness did not prevent progression to severe disease. The study findings do not support the use of ivermectin for patients with COVID-19.

[msandford]

On the one hand, you've made a rebuttal, quoting the paper. That's good.

On the other hand, you've utterly failed to understand what I'm attempting to say. So that's less good.

> The data you're referencing is too small to be statistically significant.

I explicitly acknowledge this.

>> So on the one hand I completely agree with you on the necessity of having enough people to dodge the problem of random chance. "the law of large numbers" on Wikipedia is good.

That's the acknowledgement.

>> But that does kind of hint that there might be something there.

And here's where I'm saying "if you have these three metrics which are independently all non-significant but they're all trending in the same direction, there might be a 'there' there"

Maybe I didn't say it clearly enough to begin with. I'm not alleging that Ivermectin is COVID Jesus and we all just gotta believe in him in order to be saved. I'm just trying to point out that the data previously quoted should probably get a person's "huh, what's that about?" sense going.

[PragmaticPulp]

> Could it all be random chance? Sure! But that does kind of hint that there might be something there.

Right, which is why we have studies like this: Early studies showed similar "maybe there's something here" type results, which prompted more studies, which later showed that most likely there wasn't something there.

People also seem to have forgotten that all of the other COVID drug research has progressed significantly in the past two years. Drugs like Paxlovid have indisputably significant effects that leave no room for "maybes" like this and should be ramping up quickly. Even if we were to eventually run a study big enough to find some significant effects of Ivermectin, however small, it's already been left behind by other treatment advances.

For some reason Ivermectin sticks as a political talking point, though, so it continues to be debated to death while everyone in the medical research world has long since moved on to better things.

[s1artibartfast]

> which prompted more studies, which later showed that most likely there wasn't something there

Do you have a reference to such a study? I was not aware of any well controlled and appropriately sized studies showing a negative result, but I would be open to reading one.