|
|
|
|
|
|
by dahart
3025 days ago
|
|
|
That’s all true. Sometimes I describe NNs as fancy least squares. Interesting things are bound to happen when you have a few hundred million parameters. Just to play devil’s advocate, we don’t know yet if the model is bad or if we’re just feeding the wrong kind of data, right? These optimization algorithms are good at interpolating; they do well with new data points that land inside the multidimensional convex hull of the training data. They can fail spectacularly when the new data to inference is outside that boundary. But humans aren’t that great at extrapolation either. Maybe NNs will be good enough when we show them everything there is, maybe what we think of as conceptual understanding is just as much simple interpolation of our experiences as our neural networks...? |
|
|
Possibly - we don't know. Sure if we could train one of these imagenet type classifiers with a million or billion images - close to all known objects in the universe, it may well be able to "recognize" everything. But that still doesn't solve the problem of abstraction or meaning, much less of intuition and generalization. Humans are able to generalize to new domains based on internal models of the world around us. The models used in RNNs for encoding word embeddings, seem to a bit closer to representing meaning. I agree though that NNs are evolving - we are in very early days, and who knows the NNs of the future may reveal that what we consider as understanding is no more than simple interpolation, as you suggest.