[smnrchrds]

If the COVID-19 pandemic has taught me anything about science, it's that if you do a "meta-analysis" without reading each paper carefully and critically, you can end up proving anything regardless of its veracity. You cannot replace domain-expert scientists spending huge amounts of time painstakingly going over every detail of many papers to weed out mistakes and fraud in order to write a meta-analysis, with a computer program. Well, at least not until we figure out AGI. Until then, it would be irresponsible to rely on such a program for any clinical decisions.

https://news.ycombinator.com/item?id=29249686

https://astralcodexten.substack.com/p/ivermectin-much-more-t...

https://ivmmeta.com/ (Note the long list of things in the right sidebar which the "meta-analysis" shows have huge positive effects on COVID treatment)

[Gatsky]

Yes, this is a perfect example. ivmmeta is shocking, born of either complete bad faith or madness.

[beaconstudios]

I think the generalised lesson to be learned is that in reducing complexity (and all modelling requires reducing complexity), you get to make decisions about which bits of data are cut out, and this is a decision that can introduce bias. We can't just blindly trust reductions, especially in areas where there are competing interests. This is why transparency is so important! If I can reproduce your methodology, I can critique it for bias.

[fsckboy]

ok, but... would a meta meta-analysis analysis show that on average, meta analyses do find positive benefits? get back to me.

[smnrchrds]

"TOO META"

https://xkcd.com/1447/