[Terr_]

I suspect part of the problem is that AI uncertainty and failure still doesn't always work the way humans expect, so what's "borderline" for the AI isn't necessarily borderline for human, and vice-versa. This leads to a problem when it comes to reliably detecting and escalating issues. (Especially when something slipping through might have large legal repercussions.)

Imagine a recognition task where both machines and humans have the same X% false-[positive/negative] rate, but where there's zero overlap between the cases that each system considers uncertain. Failures by the algorithm would proceed without being flagged for review, and all the human reviews would be of "obvious" things.