[allenz]

It's theoretically possible for a neural net to do this; the network just needs to have the explanation as an output. I agree that decision trees would be more reliable and easier to train, but I'm not sure if hardcoding every feature is scalable.

[function_seven]

How do you know that the explanation jives with the other outputs, though? It seems like a turtles-all-the-way-down situation, because now I want to see how it was properly introspective of its own decision making.

Also seems like it’s another magnitude of complexity in the neural net to have it not only train and learn on your inputs, but also train and learn on its own training and learning.

[allenz]

Neural networks don't do anything as sophisticated as self-referential introspection. They just fit the outputs you train them with. The training data you provide would have to include the desired explanations.

Consistency is enforced by the dataset, and also by the model. Both outputs would read from the same hidden layer--the one that encodes the desired transformation.

[snaky]

>How do you know that the explanation jives with the other outputs, though?

The third neural net would do the checking, obviously.

[randomsearch]

> the network just needs to have the explanation as an output

And how would you evaluate whether the explanation was correct or not?

[Houshalter]

You give it explanations as training data and it tries to predict them.