[baxtr]

I think you have captured the essence of it. What I wonder though: have systems before been without bias? Is the DL/ML bias worse than the one we had before?

[iamnafets]

For ML systems, it's an engineering mistake to deploy a complex model when you don't have a simpler baseline (e.g. does this outperform a basic n-gram model?). Similarly, it's a strategic mistake to deploy a deep learning model without assessing the baseline of human performance (including bias).

I see the problem of inexplicability as less salient than (1) responsible, informed deployments of models, and (2) ongoing measurement (especially against a human baseline).

You can deploy explainable models without (1) and (2) and end up with a much, much worse result.

[ACow_Adonis]

The important phrase there is engineering mistake.

Intelligibility and ongoing responsible measurement creates a performance metric, and a line of responsibility.

To many, especially if they receive large pay but are incompetent and/or face legal risks if found liable, these are significant benefits.

/depressing, I know...

[_emacsomancer_]

But if the logic before (for, say, whether to loan people money) was some sort of flowchart or checklist or whatever, it may be bad, but it's inspectable, and so could be examined, evaluated, and changed. The DL/ML creates effectively uninspectable black boxes.