[pitchups]

>maybe what we think of as conceptual understanding is just as much simple interpolation of our experiences as our neural networks...?

Possibly - we don't know. Sure if we could train one of these imagenet type classifiers with a million or billion images - close to all known objects in the universe, it may well be able to "recognize" everything. But that still doesn't solve the problem of abstraction or meaning, much less of intuition and generalization. Humans are able to generalize to new domains based on internal models of the world around us. The models used in RNNs for encoding word embeddings, seem to a bit closer to representing meaning. I agree though that NNs are evolving - we are in very early days, and who knows the NNs of the future may reveal that what we consider as understanding is no more than simple interpolation, as you suggest.