[Houshalter]

That's exactly my point. Spiders can't learn. Maybe they are amazing for what they do, but the comparison is apples to oranges. They do fundamentally different things.

Of course spiders are complex. Everything in biology is complex. It's been slowly refined over millions and millions of years. Countless trillions of tweaks to its DNA tested and rejected until it hit upon something that worked. No human could design something as complex as a spider.

Our artificial neural networks are just as complex though. AlphaGo probably knows millions of facts and heuristics about Go. You would never be able to understand everything it knows. Even though we didn't create the complexity ourselves, we "evolve" it just like the spider, sometimes even with algorithms based on evolution. But we have much more efficient algorithms than that, like backprop.

[YeGoblynQueenne]

>> Our artificial neural networks are just as complex though.

I don't know about that. I think it depends on how you count the complexity of a model. There's three cases: a) insects are less complex than machine learning models, b) both are equally complex, c) insects are simpler.

If (b) or (c) is the case then we're shafted, because we're failing to reproduce insect behaviour despite building systems at least as complex as theirs. If (c) is the case, we're particularly screwed because it means we're missing a simple, more efficient way to do things.

If (a) is the case then we may have a hope to get to where insects are in terms of complexity and possibly reproduce their behaviour. Still, there's the issue of combining models to perform well at a broad range of tasks, which is far from solved.

[Cybiote]

The large majority of deployed machine learning algorithms actually can't learn. The bias is so large that most (all?) specialized hardware assumes that only inference will be done. In contrast, there is evidence for online adaptability in insects^

^ http://www.annualreviews.org/eprint/5PHddkgYYKaduPp4CcN5/ful...

[YeGoblynQueenne]

>> The large majority of deployed machine learning algorithms actually can't learn.

Learning algorithms construct a model of data- that's what we call "learning", or "training". The trained model is then used to make decisions, so that's what you would deploy in a production system.

In that sense the deployed systems don't learn any further, yes, but that's because you don't deploy the algorithm per se.

An exception is what is known as "online learning" algorithms, that can continuously update their model as new data becomes available (hance, "online").

[Cybiote]

I took care to point out deployed models. I did not say the models did no learning. In fact, one can draw an analogy from evolution and instinct to offline training/hyperparameter search/architecture tweaks and deployed model.

And, technically, the manner by which most nets are trained, via (minibatched) stochastic gradient descent is online. But the architectures are so ill-adapted to learning continuously and backprop is so data inefficient that retraining on new data is often just better done from scratch (or at best, freezing weights).

[Houshalter]

They learn just fine during training.

[bbctol]

In terms of number of nodes, complexity of the arrangement of nodes, and amount of computation performed by each node, even a small biological brain is incalculably more complicated than a top ANN.