"The size of this increase on IQ test score correlates highly with the methodological concern of whether a study used active or passive control groups. This indicates that the medium effect size is due to methodological problems and that n-back training does not increase subjects’ underlying fluid intelligence but the gains are due to the motivational effect of passive control groups (who did not train on anything) not trying as hard as the n-back-trained experimental groups on the post-tests.
The remaining studies using active control groups find a small positive effect (but this may be due to matrix-test-specific training, undetected publication bias, smaller motivational effects, etc.)"
This is really great! I'll have to look into this when I have a little while. Thanks for passing it on. Just on a glance, I disagree that it really shows much of an increase though. Just glancing at the forest plots of those effect sizes tells me that this isn't a very strong effect at all (if it's there).
And there is also the so called 'file drawer' effect. I was at one of the big cog psych conferences a few years ago when a colleague was asking around -'Do you have any failed to replicates for WM training?' Everyone was so excited with the original Jaeggi 2008 paper, went out and tried it, and had a tough time replicating what turned out to be a severely flawed study.
Not to say that I'm not open to the idea. I'd love for it to be true, I just think large-effect-size effects are not often mired in the controversy that this one is. They're hard to find.
Because JumpCrisscross's post [0] implies that w1ntermute cherry picked a component of the Gwern meta-analysis to prove his point while the next paragraph refuted it.
"The size of this increase on IQ test score correlates highly with the methodological concern of whether a study used active or passive control groups. This indicates that the medium effect size is due to methodological problems and that n-back training does not increase subjects’ underlying fluid intelligence but the gains are due to the motivational effect of passive control groups (who did not train on anything) not trying as hard as the n-back-trained experimental groups on the post-tests.
The remaining studies using active control groups find a small positive effect (but this may be due to matrix-test-specific training, undetected publication bias, smaller motivational effects, etc.)"
http://www.gwern.net/DNB%20meta-analysis#analysis