[beanjuice]

Beautiful. We are in a golden era of citizen science, where access to knowledge, tools to connect, tools to process data, and the ability to communicate this is at an all time high. The kind of stuff you see on Youtube is amazing: people like AppliedScience achieving incredibly things in the garage, or recently NileRed took a nature paper [0] 1 step further and published it on youtube [1]).

From a chemist/material scientist perspective: Whether the results of the Riff trial may ever have a p value suitable for nature/science, likely not. When it comes to the human body and our biology, a mass trial like this may even be more useful than traditional studies, where pre-existing biases in data collection may weed out the most useful 'Riff'. Better than that, the information collected by mold_time is regularly released and discussed, in the open, on twitter/x [2].

[0] https://www.nature.com/articles/nature25476

[1] https://www.youtube.com/watch?v=CglNRNrMFGM

[2] https://twitter.com/mold_time

[bumby]

>Whether the results of the Riff trial may ever have a p value suitable for nature/science, likely not. When it comes to the human body and our biology, a mass trial like this may even be more useful than traditional studies

Can you explain this further? I'm curious about it, because at face value it seems like it is somewhat contradictory. On one hand, you're saying it won't likely be demonstrably significant enough to generalize, but then you say it will be more valuable. Are you saying it's value is in it's non-generalizability? I.e., each person finds what 'riff' works for them? I thought the point of publishing results was, in part, to separate the wheat from the chaff so we don't all have to run a ton of self-experiments.

[beanjuice]

p values are meant to relate whether or not 'some' hypothesis is statistically relevant versus a control study. For something like biology, human nature, dieting, weight loss, in a world where you can't really control how stressed someone is, how far they walk to work, among 100 other things... if you're trying to find out how 'the potato' works, safe citizen science (a Riff trial) may be far more effective than 'traditional' proofs to get to the bottom of something that works, and then someone can do a controlled study.

[bumby]

>if you're trying to find out how 'the potato' works, safe citizen science (a Riff trial) may be far more effective than 'traditional' proofs to get to the bottom of something that works

I don't know that I agree. Because the world has an infinite number of variants to "the potato" but an individual has a finite amount of time to try them. So while I agree that there are a lot of confounding factors in biology, we still need some way to triage our efforts. Analyzing the results of controlled studies seems like the best way to do so, even if it's not guaranteed to work for a specific individual.

What I suspect will happen is that people will end up wasting too much time on endless "potato" variants that don't work, and they would have had better results on focusing on those controlled studies that show promise.* For every person that finds a diamond in the 'riff', there's probably countless people that found nothing (or worse, detrimental effects). We also have other tools, like meta-analysis, to help. That seems more effective that "throw everything at the wall and see what sticks".

* you can see this in the supplement space. There are people who are constantly spending money searching for the secret new supplement hack when they'd be better off focusing on the things that science has shown to consistently work. They end up majoring in the minors, as it were

[beanjuice]

You're correct on all accounts, save for my optimism. Maybe the Riff can only work for something of this style when data feedback losing 10 lbs is large, compared to supplements, and the confines of the experiment are relatively limited (30 days). In hindsight, my comment was supposed to be 'its exciting that random people can get together and try to figure something out', and less about the difficulties of biology in science.

[axiomaticdoubts]

I don't think mold_time is a good example of citizen science done right. They have said multiple false things and refused to address or correct their statements when other people pointed them out. See https://www.lesswrong.com/posts/NRrbJJWnaSorrqvtZ/on-not-get... and https://www.lesswrong.com/posts/7iAABhWpcGeP5e6SB/it-s-proba...

It's been more than a year since I alerted them of the multiple falsehoods in A Chemical Hunger, repeatedly, and they haven't done anything about them.