Hacker News new | ask | show | jobs
by guygurari 5502 days ago
While I understand your thinking, what you're suggesting is not the correct methodology. General Relativity makes a specific, quantitative prediction for an effect. The prediction is not trivial, in the sense that the most reasonable alternative theory -- Newtonian dynamics -- predicts there should be no effect at all. (This is the null hypothesis if you will.)

The GP-B experiment measured this effect and found agreement with the GR prediction. This does not prove that GR is correct; rather, it is a piece of evidence that implies GR is more likely to be a correct description of gravity than what we previously believed [1]. Because the prediction was quantitative, it is unlikely that the result is caused by something else, which makes the evidence in favor of GR that much stronger.

Now, control groups are often used in life sciences fields. For example when you test a drug, you have a control group that takes placebo. It's not my field but as far as I understand this is done for two reasons. First, there is no quantitative prediction regarding how effective the drug should be, because drugs are not understood so precisely. So the prediction you're testing is much weaker; it's just a boolean. Second, there is a known effect -- the placebo effect -- that can affect results. In other words your null hypothesis is that there may be some effect. These things mean that, without a control group, the evidence in favor of a drug's effectiveness is not very strong.

[1] That is not to say that we believed GR was wrong, but we can never be 100% sure, and every piece of positive evidence strengthens the case.

3 comments

Right, the efficacy of GR was being tested, and the control was Newtonian dynamics. GR was found to successfully predict the results observed. The control is to attempt to predict the results with Newtonian dynamics and check that that prediction is less accurate.

If you accidentally did an experiment where GR and Newton predict the same thing, the control would kick in and tell you that you hadn't proved anything.

I applaud the perfect precision of the above explanation. One so rarely sees perfectly precise explanations these days.
Beautiful explanation. One thing, though:

  > Second, there is a known effect -- the placebo effect --
  > that can affect results. In other words your null
  > hypothesis is that there may be some effect. These 
  > things mean that, without a control group, the evidence
  > in favor of a drug's effectiveness is not very strong.
Placebo effect is but one confound variable you'll encounter in pharmacological and all other "non-quantitative" prediction experiments. There are plenty of others; for instance, order of measurement, influence of experimenter, subtle pre-existing differences, and so on.

In order to keep confounds in check, scientists attempt to keep everything either equivalent (by matching samples as precisely as scientifically feasible) or randomly distributed (by using, for instance, Latin squares and other randomization techniques).