|
|
|
|
|
|
by Houshalter
3257 days ago
|
|
|
I recall a quote from some neuroscientist. That whenever he hears people say "we know nothing about the human brain", he wants to smack them with a 900 page neuroscience textbook. I think the biggest issue is these domains are isolated and don't talk to each other. It's not that ML researchers couldn't be inspired by neuroscience research. They just don't know any. I talked to a researcher outside of the mainstream who was obsessed with biologically plausible models. He got good results, but not SOTA. However his main argument was that his methods were much faster and more data efficient than standard practice. E.g. they did online learning and didn't suffer from catastrophic forgetting. Didn't require supervision and labelled data. Standard methods are optimized towards getting the most accuracy on benchmarks, and not necessarily under realistic conditions. Real brains don't get to save huge dataset and iterate over them later. They need to learn in real time and without forgetting previously learned knowledge. Given just a stream of unlabeled data. ANNs can't do this at all. Some biologically inspired models claim to be able to do this well. |
|
|
It's more than just that. Real brains need to optimize their internal data for action. The ultimate test of whether you've represented the world correctly is: but can you do stuff? Can you control an inverted pendulum (to pick a task) while constrained to have your representation updates be Lipschitz functions, with a Lipschitz constant based on your actuators' state (ie: speed, angle, force, etc)?
Note the trick here! Your representation doesn't have to be reconstructive itself (allowing for you to conditionally simulate only Lipschitz transformations), but the updates you perform on that transformation from sensory reafferant signals do need to change only at a bounded rate, because the physics of the thing you're moving actually have that property.