[Mithriil]

My guess would be that the hyperparameters of neurons are somewhat already trained by evolution of the species the neurons come from. I assume that neurons are defined by their genetic and environment. Then if one reconstructs the environment in which neurons normally dwell, then everything is set for this neuron to perform as it should be.

But on the other hand, one could interpret the environment variables as the hyperparameters. If you overheat the chips, it is possible the neurons might act differently (as would a brain?) If you overfeed them with, say, creatine, it might be possible that the neurons will perform erratically.