[ryanferg]

All of these brain training products are suspect. Evidence for far transfer (training in one task transferring to a different domain task) is surprisingly hard to find, and empirical findings otherwise tend to disappear or diminish when replicated.

Many of the pro-brain-training camp have already begun to shift the goal posts. First it was 'simple games increase IQ,' which turned out to be difficult to prove when well controlled studies were performed. Now it's more along the lines of 'These simple games might have preventative effects against age related declines!,' which is an even harder claim to actually prove given the difficulties performing well controlled studies on aged participants.

In the cognitive science world, if we discovered a solid far transfer paradigm, especially one which transferred to something like G(eneral Intelligence), it would be our anti-baldness pill\flying car\4-day cellphone battery. People thought that these working memory transfer effects were the real deal and got very excited about it, money poured in, and the water got muddied by all these scientists with conflicts.

I obviously don't put much stock in working memory training. I wish it worked like they said, but I don't think it does. If far-transfer shows up at all, it's tiny, and doesn't persist after delay.

[bberrry]

I was under the impression that Dual-n-Back[0] had real benefits. Is that not the case?

[0] http://brainworkshop.sourceforge.net/

[cant_kant]

From a meta-analysis:

" The 20 studies included here were all completed between 2008 and 2013....Sample sizes of treatment groups varied between 7 and 36 participants, and control groups between 8 and 43"

"net effect of n-back training on Gf outcome measures, about the equivalent of 3–4 points on a standardized IQ test"

ie: very small groups, tiny effect. Sounds dubious to me, like much research in the social "sciences"

Improving fluid intelligence with training on working memory:a meta-analysis http://scottbarrykaufman.com/wp-content/uploads/2014/08/au-e...

[freehunter]

Considering the standard deviation for IQ is 15 points, I wouldn't drink an extra glass of water per day for another 3-4 IQ points, let alone play a training game for any extended amount of time. 3-4 points is statistically meaningless.

[cant_kant]

"Considering the standard deviation for IQ is 15 points"

The standard deviation for the distribution of IQ is 15 points.

The standard deviation of the errors of measurement associated with measuring IQ is 3 points.( ie: the standard error of measurement aka SEM)

I would be willing to have 3 fingers of my left hand ( not including my thumb ) amputated to improve my IQ by 1 standard deviation.

For more on Standard Error of Measurement, see http://www.csus.edu/indiv/b/brocks/Courses/EDS%20245/Handout...

[corndoge]

What is your point? The parent made no error in his language. The inferred meaning of 'standard deviation' in this context is 'standard deviation for distribution'. The parent referred to an improvement of 3-4 IQ points, not standard deviations.

[cowsandmilk]

> The parent made no error in his language.

The parent claimed a 3-4 point in an individual's IQ is statistically meaningless by point to the standard deviation of IQs for an entire population. That is either (a) misguided or (b) intentionally misleading.

The standard deviation of a non-identical population has no relation to the statistical significance of a change for an individual.

Let's say the standard deviation of heights for males is 2.8 inches; that is what some of the internet claims. Let's use two standard deviations as statistically significant. That means, if someone woke up one day and was 6'3" instead of 5'10", that was not a "significant change" because they only changed in height by 5 inches.

Standard deviation for a non-identical population is completely unrelated to the significance of changes for an individual.

[foota]

I believe the point is that standard dev for distribution is complete meaningless when talking about measuring a person's iq and increase thereof.

[cbd1984]

Out of respect for people who don't like being surprised, that link is a PDF.

[cbd1984]

That link is a PDF.

[meowface]

One standard deviation is fairly significant, though.

I think a lot of people would do mind training once per day for a possible 3-4 IQ increase, honestly. I would.

It's also possible that it only results in a fairly small general IQ increase, but a larger increase in some specific facet of recall or cognition.

[simonster]

It's also possible that it results in an IQ increase only for a specific subpopulation that you may or may not be a part of, e.g. the linked metanalysis says that "international studies tend to find more transfer than U.S. studies" and it does not seem as though the effect for the U.S. studies was significantly different from zero.

[bostonpete]

How can you claim a quarter of a std dev is statistically meaningless? Seems to me that's dependent on the confidence intervals involved not the standard deviation.

[lmm]

If you're of average IQ, an improvement of 0.2 standard deviations is about 8 percentiles of the population - the difference between being 15th in the class and being 12th or 13th. May not sound like much, but if we assume that translates directly into income (i.e. people of 10th percentile intelligence earn 10th percentile salaries) then that's the difference between a salary of $52k and $63k.

At the high end it's even more pronounced as the tail thins. Going from +2SD to +2.2SD is the difference between 98th percentile and 99th.

[notahacker]

Bearing in mind how loosely IQ and income are correlated IRL, and the opportunity cost of brain training vs devoting the equivalent time to job-specific learning, I'd say it's far from clear a 0.2 SD boost in adult IQ should lead to any change in income at all.

If it was 1 SD, the difference in ability would be rather more difficult not to notice.

[thaumasiotes]

0.2 standard deviations is a very large effect.

[simonster]

In the metanalysis, Hedges' g is 0.24. Hedges' g is a less biased measure of Cohen's d, and Cohen d between 0.2 and 0.3 as a "small" effect (https://en.wikipedia.org/wiki/Effect_size#.22Small.22.2C_.22...). Of course there are plenty of caveats to assigning importance to effects of different magnitudes without regard to what is being measured, but this isn't a very large effect.

[thaumasiotes]

Here are a few ways you might be interested in measuring effect size:

- if the environment changes such that this effect becomes operative, what sort of change should I expect in terms of z-scores? ("how does the effectiveness of damping sound with crumpled paper compare to the effectiveness of mining with dynamite?")

- if I aim to change a quantity using this effect, what sort of change can I expect relative to the existing known ways of changing the quantity in question? ("How many laborers could I replace with one bundle of dynamite while ending up with the same size of hole?")

- if I see a change of so many standard deviations in some intangible variable, (a) what sort of effect will I see further down the pipeline in the variables that I really care about, or (b) is that amount subjectively worth the effort? ("If I have $600,000, can I make a bigger hole by hiring and outfitting diggers, or by buying and detonating dynamite?")

You're insisting on the first of those questions and only the first. The comment I responded to is explicitly phrased in terms of the third question, and 3-4 IQ points is quite significant in terms of tangible knock-on effects. It's also worth noting that an intervention yielding 3-4 IQ points is staggeringly large in terms of question #2, losing out to curing malnutrition but beating basically everything else. It is so large as to seriously damage the credibility of the result, given what we already know about efforts to raise IQ.

[w1ntermute]

Gwern's done extensive research into this, and his meta-analysis has shown there's "a net gain (medium-sized) on the post-training IQ tests"[0, 1].

0: http://www.gwern.net/DNB%20meta-analysis

1: http://www.gwern.net/DNB%20FAQ

[JumpCrisscross]

Literally the next paragraph:

"The size of this increase on IQ test score correlates highly with the methodological concern of whether a study used active or passive control groups. This indicates that the medium effect size is due to methodological problems and that n-back training does not increase subjects’ underlying fluid intelligence but the gains are due to the motivational effect of passive control groups (who did not train on anything) not trying as hard as the n-back-trained experimental groups on the post-tests.

The remaining studies using active control groups find a small positive effect (but this may be due to matrix-test-specific training, undetected publication bias, smaller motivational effects, etc.)"

http://www.gwern.net/DNB%20meta-analysis#analysis

[ryanferg]

This is really great! I'll have to look into this when I have a little while. Thanks for passing it on. Just on a glance, I disagree that it really shows much of an increase though. Just glancing at the forest plots of those effect sizes tells me that this isn't a very strong effect at all (if it's there).

And there is also the so called 'file drawer' effect. I was at one of the big cog psych conferences a few years ago when a colleague was asking around -'Do you have any failed to replicates for WM training?' Everyone was so excited with the original Jaeggi 2008 paper, went out and tried it, and had a tough time replicating what turned out to be a severely flawed study.

[ryanferg]

Not to say that I'm not open to the idea. I'd love for it to be true, I just think large-effect-size effects are not often mired in the controversy that this one is. They're hard to find.

[knowaveragejoe]

Why is this downvoted? I was under the impression that Gwern did solid resarch and was well respected in the HN community.

[iaw]

Because JumpCrisscross's post [0] implies that w1ntermute cherry picked a component of the Gwern meta-analysis to prove his point while the next paragraph refuted it.

[0] : https://news.ycombinator.com/item?id=10846443

[ryanferg]

So it definitely transfers 'near'. You'll get better at the dual-n-back test. And possibly other visio-spatial working memory tests. But the question of far transfer- will it make you smarter- is probably not. Here is a link to a 2015 meta-analysis that looks at the question of working memory training (including dual-n-backs) transferring to other working memory tasks:

https://www.dropbox.com/s/084fvteji1tyz8t/2015-schwaighofer....

Melby-Lerva
g & Hulme, 2013, is also pretty damning.

[cLeEOGPw]

That's exactly, if my understanding of changes happening to brain while performing any activity in general is correct, what should happen - you get better at doing that activity. So, unless your work contains some kind of dual-n-back activity, you are not getting better at it. You get better at work by doing work, not by playing brain game.

I am really happy they got fined for this deception.

[pocketstar]

4-day smartphone* battery, I can remember when I only charged my mobile telephone once a week.

[function_seven]

I'm sure that same mobile telephone was capable of far less than a smartphone. I remember my Nokias being able to last for days on end (unless I played a lot of Snake!)

While I wouldn't trade my current smartphone for one of those, I do miss the lack of battery anxiety, and wish the manufacturers would make models for those of us who aren't obsessed with thinness. An iPhone 6 at 5/8" (15mm) would be awesome.

[spooningtamarin]

Nope, it's just bad software engineers. Africa has those pretty very battery friendly "smart phones" and they do everything that yours does, probably without the fancy games.

I've been tweaking my Android tablet's code (10" screen), profiling, setting up all of the possible GPU optimizations (whole UI is whenever possible displayed using it) and this is the result.

http://imgur.com/QLN17SS

This is about a one year worth of work but I guess Android engineers don't have time to optimize per device.

You could probably squeeze out the same for any mobile device, although iPhone is probably not that configurable.

[Tiksi]

While that is pretty awesome, and I've played with Android tweaks to get better battery life in the past, the software doesn't seem like the biggest consumer of power. These days I generally I see >80% of battery power being used by the screen, so no matter how well optimized the software/cpu/gpu is, actually using the device is going to kill the battery.

Added: With Timur's Kernel and some rtc/wakeup/governer tweaks I can get at least 2 weeks on my nexus7.

[raverbashing]

Africa's "smart phones" (whatever that means) are usually a feature-phone with a web browser. Much less capable than a smartphone

Turn off wifi and 3G/4G see how long your smartphone battery lasts. Oh I did this recently with an old phone, it lasts around 4 days as well (and that's with an old battery)

Also a lot of apps abuse smartphone processing updating how many times per minute

But screens and processing power still cost a lot of energy

[_nedR]

Really want to read more about it. You should do blog or do a write-up on it on xda-developers about it.

Hell you should release a mod for whatever tablet you are using, and make a killing.

[atratus]

amazing, write this up please!