This is a perspective I have not heard before. Iiuc, you are saying that there is a risk that the AI will come up with plausibly deniable things and just give us the rationalizations.
There are two ideas here, I think. First is that humans aren't actually very good at plausible explanations of decisions or causes. We're very good at providing acceptable justifications after the fact. So good at it, that when combined with the deep-seated human need for the world to have order, that people readily confuse the two.
Second, it may not be desirable to have AI that replicates this. Teaching a machine to give us post-hoc justifications the way we do to one another may not be as helpful as it sounds.
Second, it may not be desirable to have AI that replicates this. Teaching a machine to give us post-hoc justifications the way we do to one another may not be as helpful as it sounds.