Hacker News new | ask | show | jobs
by thaumasiotes 3818 days ago
Here are a few ways you might be interested in measuring effect size:

- if the environment changes such that this effect becomes operative, what sort of change should I expect in terms of z-scores? ("how does the effectiveness of damping sound with crumpled paper compare to the effectiveness of mining with dynamite?")

- if I aim to change a quantity using this effect, what sort of change can I expect relative to the existing known ways of changing the quantity in question? ("How many laborers could I replace with one bundle of dynamite while ending up with the same size of hole?")

- if I see a change of so many standard deviations in some intangible variable, (a) what sort of effect will I see further down the pipeline in the variables that I really care about, or (b) is that amount subjectively worth the effort? ("If I have $600,000, can I make a bigger hole by hiring and outfitting diggers, or by buying and detonating dynamite?")

You're insisting on the first of those questions and only the first. The comment I responded to is explicitly phrased in terms of the third question, and 3-4 IQ points is quite significant in terms of tangible knock-on effects. It's also worth noting that an intervention yielding 3-4 IQ points is staggeringly large in terms of question #2, losing out to curing malnutrition but beating basically everything else. It is so large as to seriously damage the credibility of the result, given what we already know about efforts to raise IQ.