[argonaut]

No deep learning researcher believes neural networks "basically replicate" the brain's function. Neural nets do a ton of things brains don't do (nobody believes the brain is doing stochastic gradient descent on a million data points in mini-batches). Brains also do a billion things that neural nets don't do. I've never even taken a neuroscience class, and I can think of the following: synaptic gaps, neurotransmitters, the concept of time, theta oscillations, all or nothing action potentials, Schwann cells.

You have missed something fundamental about how the brain works. Namely, neuroscientists don't really know how it works. Neuroscientists do not fully understand how neurons in our brain learn.

According to Andrew Ng (https://www.quora.com/What-does-Andrew-Ng-think-about-Deep-L...):

"Because we fundamentally don't know how the brain works, attempts to blindly replicate what little we know in a computer also has not resulted in particularly useful AI systems. Instead, the most effective deep learning work today has made its progress by drawing from CS and engineering principles and at most a touch of biological inspiration, rather than try to blindly copy biology.

Concretely, if you hear someone say "The brain does X. My system also does X. Thus we're on a path to building the brain," my advice is to run away!"

[hmate9]

You are right, we do not know everything about the brain. Not even close. But neural networks are modelled on what we do know of the brain. And "milking" neural networks completely means we have created an artificial brain.

[eru]

Did you just ignore the first few lines of argonaut's comment?

Recently, we also introduced activation functions in our neural nets, like rectified linear and maxout just for their nice mathematical properties without any regards to biological plausibility. And they do work better than what we had before.