6!? They used an N=6 to make such a bold claim? If I had ever thought about that as a sample size for any scientific research my professor would have run me off campus.
With 6 subjects, it feels a bit silly to even turn on the math. 5% is about two legs of a participant. I think humans can process those low numbers instantly, intuitively; we can do it by pattern recognition, without the need for abstraction. The statistics of a dinner party. I think even some animals could do some "math" on 6 food.
Should be called "collected case studies" or something.
It is a paired t-test as the test takers are the same individuals. This is confirmed in the statistical analysis section of the actual paper. Paired t-test have more statistical power because we don't correct for group variation (as they are the same group measured twice).
Edit: One way to think about this is assuming the effects are random and symmetric then the probability of going up is 1/2. If all 6 participants had increased scores, then the p-value is at least 1/2^6 and this is not taking into account the magnitude of the shift.
You're right about the paired test (although I think there's an issue with the group variance). But your final calculation is for a sign test. For a t-test where every point shifts a very little bit (e.g. 0.01 stddev), p is larger, I think.
Yes, my edit was more meant to illustrate how 6 data points can yield a significant result. Paired t-test is all about the differences. So if all test subjects shift up by 0.01 (no variance in the differences) then the statistic will actually be infinite (i.e. very large) and so p is theoretically 0 (very small). I also agree that the paired testing needs more convincing from the researchers.
Unfortunately a lot of medical science involving expensive laboratory resources and trials currently rests on sample sizes around this range (~5-20). Then whenever there's any kind of technical innovations that makes it cheaper to collect the biosignals there is a sudden series of breakthroughs in what is really happening. All science is a work in progress.
Should be called "collected case studies" or something.