Hacker News new | ask | show | jobs
by walnut_eater 1951 days ago
I'd like to point out a number of serious methodological issues with this study.

1. They tested whether the dye was tasteless on students who were not wine students, just students pulled off of the agricultural school campus. This study basically says "Group A, who aren't trained in tasting, say that there's no taste. However, Group B, trained in tasting, report a difference." (More on that "difference" below.)

2. They tested the dye as being tasteless when mixed with wine. If it's truly tasteless they should do the same test with water. I suspect they don't because it does have a taste, more in this below.

3. Even in their experiment on untrained students trying to show that it is "tasteless," the wine containing dye was in fact identified as tasting different 120/300 times (expected value 100/300 if there was no difference in taste). This is not enough evidence to show definitively whether the dye is tasteless or not. The raw evidence reported actually indicates that it is more likely to have a taste than to not have a taste.

4. The way that they asked the wine students to compare the wines is convoluted - overly complicated, as if to obscure the truth. The wine students were asked to compare wines by sorting each feature as belonging more to Wine A versus Wine B. So, if neither wine particularly has some feature, you would still be forced to assign the feature to one of the two wines. The only two wine comparisons done were true white wine versus true red wine, and then true white wine versus white wine dyed to look red. They never actually do a comparison of the white wine dyed to look red compared to true red wine, which would be the most informative comparison.

I will transition here from methodological issues with this study to my opinion on this study.

This study appears to have been designed with a specific outcome in mind, regardless of the underlying truth about people's ability to taste wine. The conclusions stated in the study are not supported by the evidence actually collected in the study. The best thing I can say for this paper is that they report their methods clearly enough for someone to be able to see that they are heavily flawed.