[VHRanger]

If the paper gets traction and the model specification is indeed not robust, the first thing you'll see in a few months is something like "paper xyz: comment" driving holes in the methodology getting even more traction. Empirical microeconomics is fairly open about methodology flaws and critiques.

Also, by the way, you see pretty much the same type of thing in a ton of fMRI neuroscience, medical and psychology studies (even the ones you'll later see on NPR or ted talks). You shouldn't ever believe any one empirical result in basically anything except maybe CERN particle physics type work.

[gone35]

Your response, and the ostensible fact my critique went completely over your head, probably shows you should definitely spend more time familiarizing yourself with the discipline, before going around defending it.

You should start with 'calibration' in economics. No, it is not quite what you (seem to) think it is. No, it is not quite "the same type of thing" as p-hacking and low-powered studies in psychology. (Whose poor reliability, by the way, is almost common knowledge by now.)

Look it up, and see how the sausage is made.

[VHRanger]

My point is that it's very easy to check "model calibration" (eg. "Plug values from outside data"). Just run the code with different values.

Because it's so easy to check those things (assuming the data is not proprietary or whatnot) I'd argue it's a much lesser problem than the "garden of forking paths" in lab experiments where it's much harder to test robustness of the result.

Moreover, I don't think anyone intelligent is foolish enough to take the coefficients in an econometric study literally; at least I would hope not.