[eli_gottlieb]

>Real brains don't get to save huge dataset and iterate over them later. They need to learn in real time and without forgetting previously learned knowledge. Given just a stream of unlabeled data.

It's more than just that. Real brains need to optimize their internal data for action. The ultimate test of whether you've represented the world correctly is: but can you do stuff? Can you control an inverted pendulum (to pick a task) while constrained to have your representation updates be Lipschitz functions, with a Lipschitz constant based on your actuators' state (ie: speed, angle, force, etc)?

Note the trick here! Your representation doesn't have to be reconstructive itself (allowing for you to conditionally simulate only Lipschitz transformations), but the updates you perform on that transformation from sensory reafferant signals do need to change only at a bounded rate, because the physics of the thing you're moving actually have that property.