Hacker News new | ask | show | jobs
by spekcular 1525 days ago
What's wrong with saying "Our detector is 99%+-3% efficient," if they are giving the output of some procedure that constructs valid confidence intervals? The confidence intervals will trap the true value 95% of time (or whatever the confidence level is). If it does what it promises to do, I don't see the problem.
1 comments

Because a 99+3=102 is not a valid upper interval bound. You cannot have >100% efficiency for a detector. Also, your expected value cannot be centered. So maybe 99+1-3 is a valid range (but I would be very suspicious if the bound includes 100%)
I agree 102% is not a possible value for the efficiency of the detector. But if the confidence interval traps the true value of the efficiency 95% of the time upon repeated sampling, what's the problem? That's all that's required for a confidence interval to be valid. Some CI constructions do in general give intervals that include impossible parameter values, but if they contain the true value 95% of the time, there's no issue. The coverage guarantee is all that matters.

(One should not confuse a CI with a range of plausible values, in other words.)

Ok, true, in that sense, it's fine. However, in 100% of cases I have observed so far (and they were far too many), it means that the person who gives such a result used sqrt(counts) as the error estimate, and that's not correct -- not only for the upper bound, also for the lower bound.