[corporate_shi11]

It's also my impression - from my modest exposure to DL over the past two years as a student taking courses - that deep learning must be overcome to reach AGI.

Specifically gradient descent is a post hoc approach to network tuning, while human neural connections are reinforced simultaneously as they fire together. The post hoc approach restricts the scope of the latent representations a network learns because such representations must serve a specific purpose (descending the gradient), while the human mind works by generating representations spontaneously at multiple levels of abstraction without any specific or immediate purpose in mind.

I believe the brain's ability to spontaneously generate latent representations capable of interacting with one another in a shared latent space is functionally enabled by the paradigm of neurons 'firing and wiring' together. I also believe it is the brain's ability to spontaneously generate hierarchically abstract representations in a shared space that is the key to AGI. We must therefore move away from gradient descent.

[mantap]

Don't forget the human brain takes about 7 to 8 hours off every day to rejiggle itself, to use a scientific term. The brain's architecture is better than having a training stage but it's by no means able to continually learn without stops and starts.

[rckoepke]

You see this in young puppies (3-6 month old) a lot as well. They get irritable/exhausted after 15-30 minutes of training, and usually dont seem to learn anything at all during the training activity itself. Then they pass out ("nap") for 30 minutes and when they wake up they do the trick/skill perfectly.

Same thing as humans, just more obvious/visible.