Ask HN: How do I evaluate a scientific study?

[cjw3]

Recently I’ve been trying to base more of my decisions on published literature rather than secondary and tertiary sources, mostly in the area of health and diet. But apart from sample size, I don’t know where to start in evaluating the merits of a study. When comparing existing results to make a decision, what sort of factors should I consider? Experimental design? P-value and significance level?

It’s particularly an issue when so many issues have contradictory studies, and it’s hard to decipher which one is more likely to be correct!

[sethammons]

Nobody knows anything and so many things are in dispute.

Listen to your body. Take up an exercise regiment.

Expanding: we've learned that excess sugar is likely really bad, and a bunch of other stuff. Learn what you do good on. Spend some time doing vigorous exercise (like, panting, sweating, want to lie down and stop type vigorous). After you get used to that, do an elimination diet. Go to lean meats, eggs, green veggies, and water only. Plenty of water. Do that for a few weeks. Notice how your body feels while exercising. Then, spaced out (like a week between each step), re-introduce other foods. Roots, nuts, legumes, dairy, pasta, bread, etc. Try to avoid sugary stuff, but do re-introduce it like everything else.

Most people find that different foods make them literally feel gross or bad, especially during working out. You'll learn what makes you bloat or gain weight. Find the things that don't fit for you, and just leave them mostly out of your future menus.

Vitamins? Protien? Supplements? Same process. Introduce and listen to how you feel. You'll then know more about you than any flawed study will.

[sn9]

It generally takes biology students at the advanced undergrad to early grad student level two years of weekly literature reviews with groups of their peers and mentors where they discuss the merits and shortcomings of research papers before they become able to reliably be able to do this on their own.They start out spending several hours per paper to dive deep before taking less than an hour (my professor took 30 minutes) when they're proficient.

For a layman to try to do the same without the same feedback mechanisms is an exercise in futility that will almost certainly result in misunderstandings. Most people online without that necessary background who claim to accurately interpret research literature are fooling themselves and the people who believe them.

If you want to have a more informed and evidence-based perspective on some field, you should start with the resources that are tested on millions of students every year, constructed in a coherent fashion to communicate the more reliable findings of a field that are considered essentially true by consensus: I am talking about textbooks, of course.

And if you can't be bothered to seriously read and learn the contents of thousands of pages of textbooks, how much can you say you really want to understand the context of research papers that assume that knowledge implicitly?

If the above seems to be more work than you care to commit (a completely reasonable conclusion!), just stick to some basic heuristics that seem to have been held true for decades or longer and are consistent with the advice of medical professionals over the ages.

Michael Pollan's advice to "Eat food. Mostly plants. Not too much." is pretty solid.

If you want to go further, you could try to max out your RDAs for all your micronutrients from a diet composed of whole foods by playing around on sites like Cronometer.

For more evidence-based health recommendations centered around lifestyle changes, the Barbell Medicine Youtube channel is pretty solid.

[ajaxaddicted]

When you are looking at health related studies, look for the following magic keywords: "Double-blind, placebo-controlled randomized study"- ideally, not funded by the meat, egg or diary industry. The more plant based whole organic foods you eat, the better. If you don't feel in an optimal state - don't guess, dont listen to "your body", dont follow random advice from people on the internet - find people who a actually medical professionals and(or) certified nutrionists. Do a blood, SIBO, gut microbiome tests to see where you stand.

[jmnicholson]

https://www.statnews.com/2018/12/13/solving-fake-news-proble...