[bumby]

>Whether the results of the Riff trial may ever have a p value suitable for nature/science, likely not. When it comes to the human body and our biology, a mass trial like this may even be more useful than traditional studies

Can you explain this further? I'm curious about it, because at face value it seems like it is somewhat contradictory. On one hand, you're saying it won't likely be demonstrably significant enough to generalize, but then you say it will be more valuable. Are you saying it's value is in it's non-generalizability? I.e., each person finds what 'riff' works for them? I thought the point of publishing results was, in part, to separate the wheat from the chaff so we don't all have to run a ton of self-experiments.

[beanjuice]

p values are meant to relate whether or not 'some' hypothesis is statistically relevant versus a control study. For something like biology, human nature, dieting, weight loss, in a world where you can't really control how stressed someone is, how far they walk to work, among 100 other things... if you're trying to find out how 'the potato' works, safe citizen science (a Riff trial) may be far more effective than 'traditional' proofs to get to the bottom of something that works, and then someone can do a controlled study.

[bumby]

>if you're trying to find out how 'the potato' works, safe citizen science (a Riff trial) may be far more effective than 'traditional' proofs to get to the bottom of something that works

I don't know that I agree. Because the world has an infinite number of variants to "the potato" but an individual has a finite amount of time to try them. So while I agree that there are a lot of confounding factors in biology, we still need some way to triage our efforts. Analyzing the results of controlled studies seems like the best way to do so, even if it's not guaranteed to work for a specific individual.

What I suspect will happen is that people will end up wasting too much time on endless "potato" variants that don't work, and they would have had better results on focusing on those controlled studies that show promise.* For every person that finds a diamond in the 'riff', there's probably countless people that found nothing (or worse, detrimental effects). We also have other tools, like meta-analysis, to help. That seems more effective that "throw everything at the wall and see what sticks".

* you can see this in the supplement space. There are people who are constantly spending money searching for the secret new supplement hack when they'd be better off focusing on the things that science has shown to consistently work. They end up majoring in the minors, as it were

[beanjuice]

You're correct on all accounts, save for my optimism. Maybe the Riff can only work for something of this style when data feedback losing 10 lbs is large, compared to supplements, and the confines of the experiment are relatively limited (30 days). In hindsight, my comment was supposed to be 'its exciting that random people can get together and try to figure something out', and less about the difficulties of biology in science.