[tristanho]

The results are what you would expect if you measured two identical fonts, ie random.

You can make the _claim_ in your comment, but there isn't statistical evidence that this effect is more than random, or if the same people were tested again their results wouldn't be opposite. Put another way, it doesn't pass the null hypothesis.

[thorum]

I think the best approach would be a graph showing the distribution of users across different WPM differences, and if possible a second experiment with two non-BR fonts (to see how much random variation is expected). The positive results might all be noise, but I don’t think that should be the default assumption.

[tristanho]

FWIW, the article does include some data on the distribution, as well as address your exact objection:

> Since posting this experiment, I've received a lot of side comments along the lines of, "Well, of course I don't expect Bionic Reading to work for most people, but for [my subpopulation], it really works." If that were the case, we might expect to see disproportionate benefits for those participants who read faster with Bionic Reading than for those who read faster without Bionic Reading. Let's look at how many participants read faster with each font and their average speed gains.

Table 3: Summary speed differences per faster font Count Percent Delta (WPM) Bionic 998 52% 35 Non-Bionic 918 48% 43

> The number of people who read faster with Bionic Reading was slightly greater (52%) than the number of people who read faster without Bionic Reading (48%). That said, those who read faster with Bionic Reading only picked up 35 words per minute on average. In contrast, those who read faster without Bionic Reading picked up 43 words per minute. It does not appear that when Bionic Reading works, it really works.

[thorum]

Yes, but they’re still averaging the results in the “it worked” category. Hypothetically, if it worked very well for 5 users and only a little for 95 users, an average score over all 100 will make it appear to not work for anyone. I’m not going to argue this is definitely the case - just that this analysis doesn’t account for the possibility, and it wouldn’t be unexpected in the area of reading ability, where one would expect individual differences to be important.