|
> A 16 year old has a decade of vision input, knows tens of thousands of words, has a coherent theory-of-mind, and has self-preservation instincts honed over hundreds of millions of years of evolution. No they don't. Where were you driving 23 days ago? Or, if you could categorize your driving data in your head, what did you pass, precisely, in the car while you were driving 96 hours ago? AI training data has all of this. It has perfect timestamps with 25-30 frames per second (or more in some cases, with multiple cameras feeding frames in to the dataset) with full 360 view in many cases, adding up to millions of man-hours of driving data from thousands of drivers, which is longer than any human lifespan. A lot of times this data is supplemented with IR representations or LIDAR representations of the environment as well, something a human can't even see. > Similarly, driving AIs don't speak English, can't take instructions, and are simultaneously learning physics, theory-of-mind, the rules of the rode, and signage conventions without having agency during most of their training. They're not learning that on their own. They're being poked and prodded by humans in the loop (and indeed, automated tagging tools) who have tweaked the models by adding numerical weights to "bad outcomes" and "good outcomes". Or "good categorizations" and "bad categorizations", or "aligned responses" and "unaligned responses". And because it's just a dumb statistical model, if you tell it the color blue is bad and that it should resist answering any questions related to the color blue, it will agree, because its entire heuristic model for organizing the world was designed by humans. Similarly, it doesn't have a theory of mind. If it did, it would also have agency and it would already be AGI. Instead, it has access to examples of data of humans exhibiting theory of mind with other humans. And it's got enough parameters in its statistical model that it can accommodate a weighting for this "theory of mind" impact on what the expected output should be. LLMs are nothing like an intelligent child raised in a black box. I mean, we already have examples of this, vis-a-vis being both blind and deaf: https://en.wikipedia.org/wiki/Helen_Keller. |
Just because I cannot do that, still my brain had this input, and was "trained", and we know the real neurons of the brain work completely different in all regards (connecitvities forming, activations, weighting) than our gross simplification of one activation function with a weight.. so not getting your arguments at all.
I have been trained on all these inputs, and even if I cannot recall them now they manifested in a my superior brain somehow.
Nice that in theory the AI training data has all this available, still the resulting model is much simpler, and also cannot recite all of its inputs seen, too??