[osazuwa]

A few points for context:

1. Those R squared numbers in table 2 are quite low. The statistical analysis itself is painfully basic. I'm a statistician, and I've a bookshelf full of excellent books written by statisticians who publish in psychology, so they should have been able to find someone to beef up the analysis a bit. That said, psych publications is also notorious for bad data analysis. I'm an outsider to this field, but this feels like an omission.

2. The paper doesn't use any causal inference techniques despite making a clear causal claim in the title. They use simple linear regression, which is not a causal model unless causal assumptions are made explicit. In my book this is a grave sin, one avoids making causal assumptions that might be hard to defend, fits a model with linear correlation, then makes causal conclusions. To understand what I mean by causal model, read this (these techniques are well-known and if the authors' defense is that they don't know them, it reflects badly on them): https://en.wikipedia.org/wiki/Rubin_causal_model

3. It is worth thinking about what the intelligence assessment, the AFQT test, actually quantifies. One critique of AFQT is that it is more of a measure of literacy than intelligence. Read more here: https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti.... Maybe the title should be "Having parents that make sure you can read by age five has a greater effect on success than grit."

4. The AFQT and other intelligence tests have a pesky problem of having averages that change in time. This suggests that much of what they measure is not innate, but learned.

5. Social psychology is plagued by a reproducibility crisis. I'm sorry to say, their papers, no matter how provocative the findings, have to be viewed with great skepticism. https://www.psychologytoday.com/us/basics/replication-crisis

6. The paper says social economic background has a significant effect, but about the same effect on pay as intelligence. That seems to conflict with a pile of research that says that social economic background is the strongest predictor of success. To say that IQ is just as important as your parents' wealth is pretty eyebrow-raising.

[aidenn0]

I have a friend who got a PhD in psychology. She had me look over her analysis and it was bad. The two "highlights" were:

1. Her advisor told her to use a student's t-test for multivariate data.

2. When the P value insufficiently low to reject the null hypothesis, he walked her through p-hacking the data.

[BrandoElFollito]

I had the same thing with a friend who was getting her PhD in nutrition.

She was aware that the statistics were not strong, but they were horrendous.

After helping her to get at least a bit back on track I developped a strong skepticism to anything published without a rewiew of the stat methods. Which is almost everything one can read in the press.

[spzb]

I'll take your word for it on the stats - it never was my strong suit. But having read Duckworth's "seminal" book on Grit it's also incredibly unscientific, frequently mistaking survivorship bias for causation.