| > In the words of statistics philosopher Deborah Mayo - "A conjecture passes a test only if a refutation would probably have occurred if it's false". Sure, one weak result out of many doesn't pass. But not passing is a far cry from "almost guaranteed" to be spurious. > Hence - the result is equivalent as if no test has actually been performed. A result like that takes a big list of plausible correlations and distills it down. If you think even a handful of the original list items are likely to have merit, then the distilled list is useful for suggesting where you should collect more data. > Or, a more simplistic example, imagine if someone observes an asteroid and says "it might be aliens". What fraction of asteroids to you expect to be aliens? If it's one in a billion, then cutting the list by a factor of 20 is useless. If it's one in a hundred, then cutting the list by a factor of 20 is very helpful. > I feel that the same is true for "yeah, but the correlation might still be true". It depends on the original list being sufficiently plausible. You can't distill tap water into vodka. |