Hacker News new | ask | show | jobs
by maxbond 3 hours ago
If 1 in 10,000 people have a disease, then a "test" which always reports the patient doesn't have the disease will be correct 99.99% of the time. "99.99% accuracy" should be "looked at with skepticism" in that it doesn't tell you what you need to know to understand the quality of a a test for a rare disease (a classifier under conditions of severe class imbalance); at a minimum, you would want to understand it's false positive and false negative rate, not (just) it's overall error rate.

See example "A": https://en.wikipedia.org/wiki/Base_rate_fallacy

1 comments

If only stating the obvious was a fallacy :)

You appear not to have understood probability theory my friend. You will never get 100% in this universe for anything. What if "its a simulation" or "a dream" arguments ensures you never acheive 100%.

Bayes probability theory will be a good start for you.

https://en.wikipedia.org/wiki/Bayes%27_theorem

Well, I do agree that all measurements contain error, but the point wasn't that the error rate would be greater than 0% but that a single headline summary of error can't always distinguish between good and bad tests.