| > As I understand, you’d need to know all the confounding factors to account for variations? Not necessarily, the idea of these methods is to basically find natural ways to group your study pop such that confounders are matched 'automatically'. Regression discontinuity analysis, for example, should handle most confounders outside of some egregious edge cases. The idea there is find a continuous variable which thresholds people into receiving the treatment or not, and then limit your analysis to only individuals on 'the cusp' of falling above or below the cutoff: idea is that these are people who basically fell into or out of the treatment due to random chance, so they form a quasi-experimental population. If this assumption is satisfied, then the confounders will be handled as if treatments were assigned truly randomly (classic application of this method is college scholarships assigned based on SAT scores or GPA). It's hard to speak super generally about the efficacy of these methods, because it does hinge strongly on the particular assumptions of each method being satisfied, but I will say that as a whole they're very well attested to in the professional stats literature and have a long and respected history in ex. econometrics. > I really don’t trust meta analysis or statistical gymnastics to draw causal conclusions from nutrition studies given the wide array of factors that we really can’t account for. That's fair, again I can't really cosign any of the results in the world of nutrition because I have no experience with that field; just wanted to chime in with some additional context about stats methods. I will say that, if properly done, the interpretation of a causal inference technique should be pretty straight up, they're not like some black box method. For example the discontinuity regression I mentioned above is simple enough conceptually that I think it's actually pretty strong evidence, even it seems like 'gymnastics' at first glance. However you are of course free to draw whatever conclusions you like regarding the rigor of the analysis or the propriety of the methods. > They need to get their act together No doubt, I would actually point to causal inference methods as a very promising avenue for the field to improve in this way (if they haven't embraced them already). Generally dealing with questions of causality explicitly in your methods (IMO) makes it way harder to just p-hack your way to a conclusion; you have to actually think through the mechanism of action and how it's represented in your model. edit: Forgot to address one question: > What about multi-variate problems? Methods definitely exist for this, but in general multivariate stats is just harder than single variate stuff. Seems like the example about red meat vs. seed oils is more of an omitted variable concern than what I'd usually refer to as a multivariate question, and to this I'd again point out that if a quasi-experimental method is properly employed confounders should be handled (though this presumes a bunch of assumptions are met, and ideally the analyst should apply some diagnostics to check that they are). |