[Estragon]

I've been to a few talks about the analysis of the LHC data, and the methods they are using are worryingly reminiscent of high-throughput biology: a search for statistically significant outliers in oodles of data. From a statistical point of view, the problem seems even worse than for biology, because the number of events being searched is much larger.

The problem with this approach is that small deviations from the statistical model can lead to statistically significant variations. Those deviations don't have to be in the scientific model under test, they can lie in the methods of measurement. So the fact that two groups are seeing similar anomalies does not imply that they've found a solid deviation from the standard model.

I'm not saying what they've done is useless. The LHC data might lead to follow up experimental designs which test much more specific hypotheses, and are therefore much more convincing. But I do think everybody is jumping the gun a bit.