Error bars account for the known and quantifiable sources of uncertainty. They don't (can't) account for unknown or unquantifiable sources of uncertainty, such as aspects of the experimental design that were not properly accounted for or unpredicted/unmodeled interactions with other particles or forces.
Known unknowns and unknown unknowns, as Rumsfeld would put it.
About a decade ago I saw a very nice figure of estimates of the speed of light over time showing this effect. Unfortunately I haven't been able to find it since.
I don't know if I have a favorite part, although the weather forecasting/probabilistic prediction and estimated error in speed of light estimates might be.
I think error modeling doesn't get enough attention, nor does error due to model uncertainty per se, and the paper explains the consequences of those two things pretty well from an applied perspective. I also think it nicely integrates model uncertainty, error modeling, and complex systems modeling all at once.
I don't think there's anything really groundbreaking in it but I think it does a good job of explaining the importance of certain things that are often really overlooked.
Just a small comment about the speed of light. The way we now define the speed of light, measuring it doesn't really make sense anymore. The speed of light is now DEFINED as 299792458 m/s and the meter definition is based on the speed of light. So in principle, one can only measure the meter and not the speed of light anymore.
Would it be the same in a hypothetical completely homogeneous gas?
My assumption is that turbulent changes in pressure cause diffraction, causing the light to not take a perfectly straight path. I don’t know enough about physics to know if that’s right.
The speed of light in a medium is the inverse of the refractive index of that medium. It's caused by the interaction between the electrons in matter (bound to atoms) and electromagnetic fields. It does not require inhomogenities larger than the atomic structure.
Materials seem to have different refractive indices for some sort of complicated quantum reason that only relies on the inhomogenity because of materials being made up of atoms containing electrons, not an sort of macroscopic inhomogenity.
If this result turns out to be right, at least part of that confirmation will involve producing a theory which reduces down to explaining other results - either from previously unknown systematic error or interactions.
Even if you only consider statistical errors, there is no guarantee that the actual value is in the confidence interval. The graphs in this article seem to use 1σ bars which have a 68% chance of containing the result, assuming a normal distribution.
Known unknowns and unknown unknowns, as Rumsfeld would put it.
About a decade ago I saw a very nice figure of estimates of the speed of light over time showing this effect. Unfortunately I haven't been able to find it since.