[azeirah]

What always perplexes me about AI is that while the neuron models may be reasonably representative of how neurons work in brains, the connections are not similar at all.

Deep learning uses layers. All neurons in layer one connect to layer 2, connect to layer 3, connect to layer 4, and so on...

In a real brain, neurons connect all over the place. It's a bigraph. Deep learning isn't even really a graph per sé, it's a hierarchy, or a weird tree if you will.

I suspect a lot of our intelligence comes from neurons living connecting through a bigraph substrate rather than a hierarchy.

Is there any research on this? Neurons feeding back onto themselves just seems natural to me, but I don't encounter anything on it.

[fooker]

Backpropagation sort of handles this. The real reason we don't have what you describe is learning algorithms have to be efficient. So, our training/inference algorithms and 'neural' network design is all targeted to be easily representable as matrix multiplications.

Maybe with some hypothetical analog computer we could have fast training of arbitrary networks.