[huyegn]

I stated that rather than explainability, for most use cases, people just want calibrated and robust uncertainty estimates NOT low quality and uncalibrated uncertainties estimates.

Your example points to models that provide low quality uncertainty estimates, but that's not true for all deep learning models.

I believe it's these low quality uncertainty estimates that lead people to look toward "explainability" as a solution, but for the majority of use cases, I think people just want better uncertainty estimates so that they can "know when they're model doesn't know".

There are techniques now to get higher quality, calibrated, uncertainty estimates that don't suffer from the problems you mentioned and I've outlined these solutions in my posted link above.

Additionally if you're interested, there is some nice recent research from google on the subject:

https://ai.googleblog.com/2020/01/can-you-trust-your-models-...

and from oxford:

https://arxiv.org/abs/1912.10481v1

[ssivark]

Thanks for the links here and above; will take a look.

> calibrated and robust uncertainty estimates NOT low quality and uncalibrated uncertainties estimates.

Could you explain what you mean by “calibrated” and briefly summarize the essential idea behind what allows the learning of robust uncertainty estimates, if not a causal understanding?

If you haven’t already, look up work by Scholkopf, Janzing , Peters and co (over the last decade) for a justification of why causal reasoning is exactly what you want if you want to generalize across covariant earth/dataset shift (which is basically what the Google blog post is about).