[kmiroslav]

Typical neural networks have multiple layers of hundreds of thousands of nodes, which then get updated very frequently during the learning phase.

It would be suicidal for these updates to be immutable (e.g. returning brand new neurons with the updated value instead of updating the neuron that's already in the graph).