| Ah, well, yes, looks like I was right, too, though. Unless I'm mistaken (and that's possible, I've changed my opinion twice now), my example outlined above is - conceivable, and - has 85% accuracy (85 people correctly identified as liars, 85% x 9900 = 8415 correctly identified as non-liars, thus a total of 85+8415=8500 of 10k total "accurately" identified), and - still only 5% or 6% of flagged liars are actual liars. EDIT to add: And if the system is tweaked as you suggest, to very rarely fail to flag a liar: - suppose it correctly flags all 100 liars as liars - suppose accuracy is still 85%, thus 8500 people in total classified correctly - thus 8400 non-liars flagged correctly, and the remaining 1500 non-liars flagged incorrectly Now still only 6.25% (100 of 1600) of people flagged as liars are actually liars. Thus, even with the tuning you suggest, this remains. (Note to self: 1. think 2. write) |
You really have to compare precision and recall values to know if the accuracy statement holds true. You could have have 100% precision and low recall and still have 85% accuracy (meaning you could never flag someone as lying and be wrong while missing a bunch of liars and still have 85% accuracy).
but if everything is totally evenly distributed, then 85% accuracy means 85% accuracy and your first statement is correct.
The real issue is that accuracy is only one piece of the puzzle.