[mike_n]

Do human doctors have a better rate? And at what cost/throughput?

[jupp0r]

This is the base rate fallacy.

    return “healthy”;

would have a 99.9% accuracy right now.

[yellow_lead]

I think you have a good idea for an AI startup here.

[smacktoward]

For bonus points, have it return one out of every 1000 or so requests with "not hot dog."

[droidist2]

Ahh, you need to use ROC AUC or something.

[contravariant]

ROC AUC is somewhat meaningless when basically any amount of false positives is going to massively decrease the utility of the test.