[RoderickBeck]

Machine learning is by definition a black box. Nonlinear statistical models with huge numbers of parameters to be estimated.

[DeathArrow]

Not all machine learning. Neural networks.

[openrisk]

Explainability towards users is essential if there is any sort of responsibility for what the model produces.

But explainability at a deeper level is also critical towards developers, to close the feedback loop and iterate towards better models.