[visarga]

You can't directly compare biological and artificial neurons like that. Biological ones have synapses that function in a much more complex way than weights in a neural net, but are also much slower and noisy.

On the other hand, we don't have a robot body to house the model in. Without embodiment it won't be able to learn to interact with the world like us.

Thirdly, in humans, specific priors have been baked in the brain by evolution (data symmetries and efficiencies). We don't know all of them yet and how to replicate. We do rely on translation invariance for images and time shift invariance for sequences, and permutation invariance for some set and graph neural nets, but they are not all the priors the brain makes use of.

[Veedrac]

Biological neurons are complex networks of thousands of synapses, and it's definitely reasonably to say a biological neuron is not 1:1 comparable to an artificial NN neuron. Biological neurons can compute XOR[1] and some even contain loops, called autapses.

However it seems fairly reasonable to say a synapse is roughly 1:1 comparable to a network parameter, in that they seem to be doing about the same sort of weighted propagation with about the same computational power. A synapse does work very differently, and has a couple of very low bandwidth side-channels, but its main job is the same job as a network weight.

[1] https://science.sciencemag.org/content/367/6473/83

[enchiridion]

Do we really rely on translation invariance significantly? It seems more like we scan a very focused area quickly