[tedsanders]

The solution to false positives is not to artificially rate-limit testing or blind yourself to potentially useful data. It's to understand that 5% is an insufficient significant threshold when your prior belief in a correlation is low.

There are really three solutions to the problem of multiple comparisons: Either (1) you use a different threshold, (2) you use a different test, and/or (3) you correctly interpret that p=5% does not imply the effect is 95% likely.

There's absolutely nothing wrong with exploring a data set, as long as you are responsible in the conclusions you draw.