[t_mann]

We don't even need to go into the 2000's. The author openly dismisses Generalized Method of Moments (published in 1982 by Lars Hansen [0]) as a 'complex mathematical technique' that he's 'guessing there are a lot of weird assumptions baked into' it, the main evidence being that he 'can't really follow what it's doing'. He also admits that he has no idea what control variables are or how to explain linear regression. It's completely pointless trying to discuss the subtleties of how certain statistical techniques try to address some of his exact concerns, it's clear that he has no interest in listening, won't understand and just take that as further evidence that it's all just BS. This post is a rant best described as Dunning-Kruger on steroids, I have no idea how this got 200 points on HN and can just advise anyone who reads here first to spare themselves the read.

[0] edit: Hansen was awarded the Nobel Memorial Prize in Economics in 2013 for GMM, not that that means it can't fail, but clearly a lot of people have found it useful.

[MichaelDickens]

I think you are significantly misrepresenting what the author said. He didn't say he has no idea what control variables are. What he said is:

> The "controlling for" thing relies on a lot of subtle assumptions and can break in all kinds of weird ways. Here's[1] a technical explanation of some of the pitfalls; here's[2] a set of deconstructions of regressions that break in weird ways.

[1] https://journals.plos.org/plosone/article?id=10.1371/journal...

[2] https://www.cold-takes.com/phil-birnbaums-regression-analysi...

To me this seems to demonstrate a stronger understanding of regression analysis than 90+% of scientists who use the technique.

[groby_b]

> He didn't say he has no idea what control variables are

He did say exactly that.

> They use a technique called regression analysis that, as far as I can determine, cannot be explained in a simple, intuitive way (especially not in terms of how it "controls for" confounders).

That's about as /noideadog as you can get.

[roenxi]

That is unfair, he says...

> "generalized method of moments" approaches to cross-country analysis (of e.g. the effectiveness of aid)

Which is an entirely reasonable criticism. GMM is a complex mathematical process, wiki suggests [0] that it assumes data generated by a weakly stationary ergodic stochastic process of multivariate normal variables. There are a lot of ways that a real world data for aid distribution might be nonergodic, unstationary, generally distributed or even deterministic!

Verifying that a paper has used a parameter estimation technique like that properly is not a trivial task even for someone who understands GMM quite well. A reader can't be expected to follow what the implications are from reading a study; there is a strong element of trust.

[0] https://en.wikipedia.org/wiki/Generalized_method_of_moments

[t_mann]

Every statistical model makes assumptions. As a general rule, the more mathematically complex the model, the fewer (or weaker) assumptions are made. That's what the complexity is for. So the criticism 'it looks complex, so the assumptions are probably weird' doesn't make sense.

If as a reader you don't understand a paper (that's been reviewed by experts), then the best thing to conclude is that you're not the target audience, not that the findings can be dismissed.

[roenxi]

He isn't saying that, he's saying he does understand the paper and therefore the findings can be viewed with some suspicion. That is the nature of research, clear conclusions are rare because real data is messy.

> Every statistical model makes assumptions. As a general rule, the more mathematically complex the model, the fewer (or weaker) assumptions are made. That's what the complexity is for. So the criticism 'it looks complex, so the assumptions are probably weird' doesn't make sense.

This is an argument of the form [X -> Y. Y. Y has a purpose. Therefore not(Y->Z)]. It isn't valid; the fact that a criticism is general doesn't make it weaker (or stronger, for that matter). It is a bit like saying meat contains bacteria so someone can't complain that some meal gave them food poisoning. They can certainly complain about it and it is possible (indeed likely) that some meat is bad because of excessive bacteria.

[t_mann]

> He isn't saying that, he's saying he does understand the paper

He literally says 'I can't really follow what it's doing', linking to a paper that discusses some issues with instrumental variable regression (what GMM is used for).