[mschuster91]

Correct me if I'm dead wrong here, but isn't "software machine learning" taking advantage from all the neurons being "interconnected", similar to a brain? How does that work with physical (discrete?) components as in this case?

[smaddox]

There's good reason to believe that different parts of the brain are quasi-specialized to particular applications. In a sense, this applies to particular software artificial neural networks (ANN) as well, particularly if the various hyper-parameters are fixed (number of neural units per layer, etc). One of the primary advantages of software ANNs over hardware ANNs (which don't really exist yet) would be the ability to easily change the hyper parameters.

Hardware implementations of ANNs, such as might be designed based on these FTJ-based artificial synapses, would have some fixed hyper parameters, and thus would be pseudo-specialized. This disadvantage could potentially be more than compensated for by a dramatic learning speedup and power-usage reduction. Transistors are highly scaled and low power, but it takes a lot of them and a lot of time to simulate each neural unit.

On a separate note, the best-performing software ANNs don't emulate spike time dependant plasticity, which is believed to be the primary learning mechanism of the human brain. Instead, they use variations of backpropagation and gradient descent, which is almost certainly not how the human brain learns. It remains to be fully understood how the two compare at various tasks. Most likely, they will have different strengths and weaknesses, making each useful in their own right.

[cbennett]

It's far from clear that STDP is sufficient to the brain's learning mechanisms, though it is certainly necessary at some scales and stages.

The possibility space between relatively simple and insufficiently general unsupervise/clustering approaches and rigid SGD schemes is large, and probably contains the brain's true inference engine. Personally, I am excited by some of the ideas brought forward in this Bengio paper: https://arxiv.org/pdf/1602.05179.pdf

[lutusp]

At the top of the article, memristors[1] are mentioned -- as artificial synapses they play a key part in the process of adding/removing weight (influence) to various pathways between neurons. This isn't strictly speaking a digital scheme -- a given weight is an analog level that's compared to analog levels from other competing artificial synapses.

The properties of the memristors can be controlled electronically, and by having programmable and persistent resistance to current flow, they play the part of biological synapses.

This entire scheme relies on our gradually improving insight into how biological brains work. As these systems evolve to higher levels of complexity, they become a practical research basis for understanding the human brain.

1. https://en.wikipedia.org/wiki/Memristor

[anigbrowl]

Quite. It's instructive to examine modular analog synthesizers, in which it's easy and fun to model such behavior. You can build strange loops with them, which offers the possibility of hysteresis from which much else flows. Also worth boning up on integrated information theory.

[jcoffland]

Neither in software or the brain are all the neurons interconnected. There are connections between neurons which are local to each other.