[kharak]

In my mind, this generalizes to the same problem with other non-stochastic (deterministic) operations like logical conclusions (A => B) .

I have a running bet with friend that humans encode deterministic operations in neural networks, too, while he thinks there has to be another process at play. But there might be something extra helping our neural networks learn the strong weights required for it. Or the answer is again: "more data".