Hacker News new | ask | show | jobs
by d26900 1948 days ago
Then how would you decide whether a finding/study is valid or not? What is your modus operandi in that case? Is there an algorithm for this (for selecting good studies or for finding the truth)?
1 comments

Meta-studies are most useful since someone proficient in the art has taken the trouble to find and analyze all relevant papers. Often times they also publish the method they used to discover and discard papers in addition to the analysis, specifically to avoid selection bias.

Studies published in reputable scientific journals like Nature are usually not bogus, especially if they were already replicated. However applying the results to everyday life is tricky - one certainly must not assign more meaning to them than the authors did, but also probably even less than that. Remember the mantra: the experiment shows only what the experiment shows, not the great opportunities you want it to show.

Note that "nutritional science" is not a hard science, their track record is abysmal. The nearest hard science we have to that is microbiology.

As a rule, all observational studies are junk - too many hidden variables, etc. There are some exceptions to it, but you will be best served by just assuming junk. If you're not willing to discard a particular observational study at least check if the study controls for obvious hidden variables - wealth, age, sex, health level, etc. For example there were "studies" that showed red wine correlates with good health, and the coverage was that we should all drink red wine. But guess what - rich people drink red wine and live longer because rich. Controlled for wealth, the effect disappears.

Thank you, I really appreciate your input!