[FollowingTheDao]

Everyone always with this "noise"! Of course there is noise! They are looking at the "noise" and telling you what it is from! Genetics!

If you average any trial out in a large population there will be "noise", but these people who live with the "noise" are the ones affected and suffering.

Does everyone have microbiome-driven effects of non-nutritive sweeteners? Probably not, but what about the ones that do?

[mlyle]

> If you average any trial out in a large population there will be "noise", but these people who live with the "noise" are the ones affected and suffering.

If you do a trial in a large population of a drug, device, or clinical practice that does nothing- a perfect placebo- you'll see a variety of effects: statistical noise.

If you do a trial in a large population of a drug, device, or clinical practice that has an effect-- you'll measure that effect, plus the statistical noise.

You can't generally tell for any individual whether the drug helped or hurt. But you can tell that more people did well (or badly) in group A than group B.

You can't even really know exactly how big the effect is precisely: just a range of likely effect sizes.

The more things you try to measure to more precisely zero in on an effect, the greater the chance that statistical noise spuriously makes one of these look important (and the larger the effect must be to be reliably measured). https://xkcd.com/882/

[FollowingTheDao]

> You can't generally tell for any individual whether the drug helped or hurt. But you can tell that more people did well (or badly) in group A than group B.

That is what they found in this study, but the OP said it was likely "noise" and had no scientific basis for saying that. My point; saying something is "noise" is a way to look cool on HN and dismiss any finding that does not fit your world view.

[mlyle]

> That is what they found in this study, but the OP said it was likely "noise" and had no scientific basis for saying that.

It's a small finding in both effect size and statistical significance, and prior probabilities count.

Barely statistically significant findings don't change my beliefs much, because the base rate and prior knowledge matter.

E.g. if you show me a p<0.05 finding that ESP exists, I'm going to dismiss it as statistical noise-- even if the study methodology is perfect it's only 10-20x more likely that ESP works than before, and 20x my prior belief of very near 0 is still very near 0.

If you show me a p<0.05 finding that green jelly beans cause acne, after studying all colors-- I don't care at all.

Here, the commenter you replied to-- api-- suggested that the study clearly indicates that there's reason to be concerned about saccharine and sucralose. It raises a general level of concern about other NNS's, but the data is ambiguous and weak. This is a reasonable reading of the study.

[TaupeRanger]

Yikes. When someone makes a comment about "noise" in this context, they simply mean "we can't tell whether the effect observed is due to the thing we're measuring". You'd need a much larger study with totally different design to even begin to approach the question of "do non-nutritive sweeteners make peoples' lives worse or shorten their lives compared to whatever else they might be ingesting". No need for the confrontational angle.

[FollowingTheDao]

> You'd need a much larger study with totally different design to even begin to approach the question

Yes, that is my point. As the OP remarked; "The others may be “complicated” but the effect also looks close to noise. " is disregarding that data.

When one thinks of the word "noise" one hears something that bothers them, like "there is too much noise in here". This is the problem with research. By getting rid of the "noise" they will only zero in to the thing they wish to hear clearly.

[simias]

If you manage to model the "noise" and make it somewhat deterministic then it's not noise any more. If genetic variations really are the reason for these variations and some people are indeed measurably harmed by these compounds then it would be a very interesting and somewhat alarming result, but that's not what the study says or what we can conclude from it.

[FollowingTheDao]

> If you manage to model the "noise" and make it somewhat deterministic then it's not noise any more.

Yes. That is called doing science.

> If genetic variations really are the reason for these variations and some people are indeed measurably harmed by these compounds then it would be a very interesting and somewhat alarming result, but that's not what the study says or what we can conclude from it.

What would make me not think that? Maybe we should investigate it. That is also called science. Saying anything is noise only disregards the response of that part of the sample as useless. It is not useless.

It is why some of us are more sensitive to to certain diets: https://www.chop.edu/conditions-diseases/carbohydrate-malabs...

[parineum]

You know what isn't science? Not doing any of that and just going off on a study that's led to more questions (aka, science) in the comments of Hacker News.

Their study had a scope, they did the study and found some results then drew some conclusions. They also found the study wasn't large enough to draw all conclusions because of noise, something they didn't know before the study.

> Saying anything is noise only disregards...

Saying something is noise indicates that the data isn't clear enough to draw a conclusion and more specific and targeted studies are needed to draw said conclusion.

It certainly doesn't mean that they're ignoring the plight of... you.

[FollowingTheDao]

I was commenting on how the original commenter was using the term noise. Not how scientists use it.

Listen, the term noise is probably the worst term for this data. Because we don’t know if it’s noise until we examine to see if it is noise. So until we know it’s noise we can’t call it noise. It’s like you’re walking into a crowded room and you’re trying to hear one thing but there is too much “noise”. This assumes we know what we’re looking for in the first place.