[d26900]

> The constant rate of "New Study Finds Coffee Improves [insert health benefit]" articles

Yep, this has become so inflationary, but it doesn't mean that studies and findings are invalidated per se. If multiple (independent) findings come to the conclusion X, then the correlation becomes stronger and stronger. When the correlation is strong (multiple findings conclude X), then you can be reasonably certain that X is likely. Or am I wrong with this?

[DenisM]

> Or am I wrong with this?

Some percentage of studies will produce random (erroneous) results, so if one cherry-picks favorable outcomes and buries the rest an impression can be created to suit any narrative. Are they cherry-picked? I don't know. What I do know is that there is a strong demand for positive studies both the dealers to sell more stuff and from the addicts to justify their addiction.

[Ekaros]

Not to forget for the people doing studies to be able to publish and deliver something... Losing the income is quite big incentive to get out studies that at least on surface look good.

[d26900]

Then how would you decide whether a finding/study is valid or not? What is your modus operandi in that case? Is there an algorithm for this (for selecting good studies or for finding the truth)?

[DenisM]

Meta-studies are most useful since someone proficient in the art has taken the trouble to find and analyze all relevant papers. Often times they also publish the method they used to discover and discard papers in addition to the analysis, specifically to avoid selection bias.

Studies published in reputable scientific journals like Nature are usually not bogus, especially if they were already replicated. However applying the results to everyday life is tricky - one certainly must not assign more meaning to them than the authors did, but also probably even less than that. Remember the mantra: the experiment shows only what the experiment shows, not the great opportunities you want it to show.

Note that "nutritional science" is not a hard science, their track record is abysmal. The nearest hard science we have to that is microbiology.

As a rule, all observational studies are junk - too many hidden variables, etc. There are some exceptions to it, but you will be best served by just assuming junk. If you're not willing to discard a particular observational study at least check if the study controls for obvious hidden variables - wealth, age, sex, health level, etc. For example there were "studies" that showed red wine correlates with good health, and the coverage was that we should all drink red wine. But guess what - rich people drink red wine and live longer because rich. Controlled for wealth, the effect disappears.

[d26900]

Thank you, I really appreciate your input!