[bencollier49]

Wow, that was quite a read. What does seem to stand out is the danger of negative results not being published. 99 studies look at an ingredient, find nothing, don't publish. Through pure chance, the hundredth study finds a correlation, and everyone is immediately convinced that yet another ingredient is unhealthy.

[eternauta3k]

The study will fail to replicate.

[TeMPOraL]

Doesn't matter; before anyone even thinks of replicating it, it's already all over the fitness & lifestyle magazines and news columns regular people read.

[ErikVandeWater]

Yes, but how long does that take, and what is the cost of people acting on misinformation in the meantime?

[FrankyHollywood]

Not necessarily, it's just a new random pick :)

[YjSe2GMQ]

This is the big way for p-value science to fail, even if every actor is honest. It's obvious once you understand what a p-value is (it literally takes 5 seconds to understand) but nearly nobody does. Even people I've worked with at a FAANG, which you'd expect have an above average understanding of mathematics.

https://xkcd.com/882/

P values don't work at all if there's a lot of people working in a field!

[ilikeatari]

For some it might take more than 5 seconds though https://fivethirtyeight.com/features/not-even-scientists-can...