[krisoft]

> In my mind, understanding a thing means you can justify an answer.

Do you understand cats? If I show you a picture of either a cat or a dog do you think you can tell which one it is? I think most people could solve that challenge, and if pressed they could vax poetically about what makes them think it is a cat. Maybe they would mention the shape of an ear, or talk about feline grace or what have you. But is that really a “justification”? Let alone one they can “defend”? How would “defending” even work in this situation?

[hayksaakian]

You could probably teach an AI to post-hoc rationalize their decisions, the same way people do.

[taneq]

You absolutely could, and it could even end up just as accurate as human post-hoc rationalization. ;)

Self-analysis and self-interpretation is pretty clearly a key part of consciousness... I do wonder - how important to the process is the actual fidelity of the interpretation? Those people you meet who think they have deep insight into their own psyche while clearly having no clue... maybe they're p-zombies. ;)

[naresh_xai]

That’s basically the gist of explainable AI

[6gvONxR4sf7o]

The point I’m trying to make (poorly) is that i don’t think a one size fits all definition of “understanding” is useful. It’s more useful to define understanding with respect to how you can justify a thing you know.

So for the case of cats, I will understand cats at a different level from a cat biologist. I can point to features that seem catlike, and they can talk about all sorts of other scientific things that make a cat a cat.

With respect to that sciency kind of understanding, I don’t understand cats. With respect to a much looser ‘point at the features’ kind of understanding, I do understand cats.

[Cybiote]

Take entomologists, bird watchers or those who identify mushrooms. In each, there are instances that are fiendishly difficult to tell apart. If you ask an expert for advice, they'll tell what features to look for and where, sometimes not even on the item itself and some requiring specialists tools.

While explanations are far from sufficient to instantly close the gap to expertise, they provide a massive boost that you might not otherwise have found on your own. The justification comes from the fact that their explanations are a reliable source of increased performance in making fine-grained distinctions. It's further demonstrated by answers to questions like "If they are so difficult to tell apart, why make these distinctions?" or "How did they come to be so similar?".