| One thing to keep in mind is that any neural net based ML system is essentially just a mathematical function imitator. The observations in this paper are spot on in the sense that many mathematical functions can have the same subset of results (success within your training data set), but can have wildly varying behavior in the general case. This is known as overfitting, and its one of the main things that should cast doubt on any ML system's capability to reliably produce results outside of a training set. In a way, these outcomes are to a point predictable (in the sense of "the possibility exists" as opposed to "this set of weights yields these generalizability results") if you take the whole "neural network" thing a bit more straight than many academics are comfortable with you doing. The human brain, or any collection of biological neurons, is in a state of constant flux, creating different networks in order to react to stimuli in the environment and implement actions that bring us closer to achieving $goals. Who hasn't experienced an off day where the gears of your mind just aren't producing what you darn well know they should be? It's just a fundamental change in the primary set of neural tools you've got to work with that day. When the weights change, so too does the output, and the function modeled. The cerebellum if I recall correctly, actually acts as a QA like functionality built into our own minds. There's nothing magic about simulating neural networks in silicon that suddenly gets you to a more "free-of-mistakes" state besides being able to condense way more dimensions of data into the networks "sensory space" as it were. Even then though, the possibility of suboptimal functions being imitated is inescapable. Try explaining that to someone that wants to save millions on workforce, or automate safety-critical tasks without concern for the consequences though. It's amazing the cognitive barriers we can build. |