[suncemoje]

I think anything is and will be explainable. Like in the OpenAI proof, I’m sure they were able to understand the solution 100% and could even drill down and ask more clarifying questions to the model. After all, the point of science is so that knowledge can be made logically transparent. If something can’t be explained, it isn’t really understood yet — and the same applies to model outputs. The only question is how much effort it takes to surface the explanation.

[fauigerzigerk]

I think explanation is itself a rather complex concept. At what point do we consider something as explained? Usually it has to do with identifying some causal factors and their relationships so that we can intervene and explore counterfactuals. But in many cases we are forced to act on the basis of incomplete explanations (e.g. in medicine).

I think there will be regulation that requires some users of AI to provide an explanation upon request. For instance, banks could be required to "explain" why you didn't get that loan. What if the decision is based on a credit score that includes some AI prediction that ultimately relies on the entire training corpus?

The bank can give you a list of factors that play into the decision but they may not be able to explain deterministically why a very similar customer did get that loan. At that point I think we're going to resort to statistics that prove a lack of bias against certain protected characteristics, but that's not really an explanation, is it?

I think we will never get useful and complete explanations for everything that AI does. Society will just accept some explanation-like thing or proxy and move on.

[worldsayshi]

And more intelligence should give an opportunity to increase explain-ability rather than just complexity. It can potentially explain the proof at the level of the listener. Make visualizations. Etc.

[criddell]

The explanation isn't the problem, it's the comprehension.

Your dog will never understand calculus or why Fourier transforms are interesting. There's almost certainly topics that are beyond human comprehension that an advanced artificial or alien intelligence can easily handle.

[latexr]

> I’m sure they were able to understand the solution 100% and could even drill down and ask more clarifying questions to the model.

If they understood it 100%, what clarification is needed?