[alexknvl]

What people in this thread seem to be missing is that modeling and decision making do not have to be done simultaneously. You always want the most accurate model of reality as you can get, but what you do with that model is up to you.

Perhaps you think there are certain feedback loops that need to be broken (your model is usually a static representation of the reality), or perhaps you prefer low rate of false negatives / false positives, or perhaps if a model is uncertain you would like to defer to a human.

If a model predicts high probability of re-offence, you might decide to delay an action as an example, or re-examine the case in more detail.

I get a feeling that machine learning practitioners are not properly trained to recognize such subtleties, everything is a binary softmax output these days :)

EDIT: This seems to stem from the obsession with purely discriminative models that either directly model a binary function y = f(x) or model a function that returns some kind of score, which is later compared to a threshold y = f(x) > theta. Neither of these lend themselves nicely to this conceptual separation between modeling and decision making.