[cec]

> I'm no biologist, but I don't think our brains are going around trying to do a grid search for the best hyperparameters.

Is that not essentially the process of evolution through natural selection? One giant optimisation problem?

[discardorama]

But not in the same brain. Sure, evolution happened and picked the right parameters; but today's brain comes with the hyperparameters baked in (with some small amount of randomness). It is able to do all the things it can do without the luxury of parallel training and tuning.

For example: you don't need to show a baby a 1000 photos of mugs before it can tell what's a mug and what isn't. Just show it 1 example of a mug a couple of times, and from then on it's able to identify mugs and mug-like objects pretty reliably.

[modeless]

> [the brain] is able to do all the things it can do without the luxury of parallel training and tuning.

I beg to differ. Newborn babies can hardly do anything. Their brains are undergoing "parallel training and tuning" 24/7 starting even before they are born. Babies train themselves on thousands of hours of visual stimuli to gain the 3D object recognition capabilities to reliably identify objects such as mugs.

[discardorama]

Newborn babies can hardly do anything because their parameters have not been set; but their hyperparameters (things like, to use NN analogies: activation function, learning rate, etc.) are baked in.

[modeless]

Right. Evolution picked the hyperparameters, and experiential learning picks the parameters. (To first order; certainly there are some instinctual behaviors programmed by evolution and I wouldn't be surprised if some hyperparameters are influenced by experience as well.)

[rasz_pl]

One data point is perfectly fine for NN, it doesnt stop learning after first sample

https://www.youtube.com/watch?v=1GhNXHCQGsM

[sgt101]

No, I don't think so. I think that evolution is a complex system in the sense that it creates feedback loops altering the fitness landscapes it optimizes over. Organisms interact with each other through competition and predation and with their environment, for example by liberating oxygen from water. If evolution were about optimization then the current set of organisms would be "fitter" compared with dinosaurs (for example). I don't fancy my chances vs. a T.Rex, more to the point I don't fancy anything that is around at the moment's chances v. a T.Rex!

[aswanson]

But the T.Rex was a local maxima. I don't fancy a T. Rexes chance against humankind with the technology, intelligence and group organization we have. The T.Rex was optimized for physical force, but that proved useless against asteroids. Humans would have a chance against that type of threat.