[onetwotree]

17 subjects per group is extremely small.

Looking at it either from a machine learning or statistical point of view, using such a small sample is problematic.

This is the chronic issue with fMRI studies, since administering an fMRI is extremely expensive, and has led to some very difficult to reproduce results in the field.

[matt4077]

People love the "n=XX is far too little data!" argument, yet it's more complicated than that. Sometimes 600,000 is too little, yet sometimes 17 is enough.

Example: you believe a newly found plant species is toxic. You give it to 17 "grad students volunteers", while giving a placebo to 17 others. All in the first group die aa gruesome death within 20 hours. None of the others do.

Result: yes significance. (also: tenure!)

I'm not saying that this study is significant (the statistics seem to be slightly beyond my event horizon), and your criticism also stops short of an outright dismissal of the research. But sample size alone makes for a bad measure of quality. Yes, even p-values are better.

[onetwotree]

I think that a small sample size is mostly an indicator that one needs to treat the results with far greater caution.

Effect size is very important in this. To continue your grad student murder example, it's completely trivial to determine which plant a student was given, based on whether they are dead or not. It becomes trickier if you measured something a bit less cut-and-dry, such as the incidence of headaches, or variance in a few voxels of a noisy MRI.

[jacalata]

I really hope you wouldn't get tenure for a study that killed all your subjects.

[rav]

Subjects? But they're volunteer grad students!

[jacalata]

Subjects, minions, whatever you want to call them :p

[hikarudo]

The technical term for that is "effect size".

[killjoywashere]

I believe some pretty fundamental fMRI spatial autocorrelation functions have been called into question as well (1). Sounds a bit like PowerPoint's autocontent wizard.

(1) http://www.pnas.org/content/113/28/7900.abstract

[matt4077]

Oh, but the paradigm-shifting revolution of a paper, sorry: "poster" surely is "Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon": http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf

[onetwotree]

I am extremely stoked that a huge sample of 17 people have decided to upvote my comment ;-)