[munificent]

It's scary in both directions.

If you let it give out tons of false positives, then patients are trained to ignore it when it cries wolf.

If you dial it back so that it gives out fewer positives, then now it starts giving out false negatives and not helping sick people.

[KingMob]

Heh, Signal Detection Theory strikes again! This problem is as old as detecting whether a radar blip is a WW2 bomber on its way or not.

Sadly, there's no perfect threshold when the signal and noise distributions overlap substantially, just different trade-offs.

(Love CI, btw!)