Hacker News new | ask | show | jobs
by YeGoblynQueenne 623 days ago
Well the majority of trained neural network models fail to solve most problems they're tested on, also. When you pick up a neural net paper (or really any machine learning paper) and look at the results of experiments the authors will invariably list the results of their best-performing model, chosen by careful cross-validation over multiple random seeds.

The point of this testing is to estimate the true learning ability of a system by looking at what it can do in the best case scenario. Nobody reports the worst-case scenario, otherwise deep neural nets would look a lot less impressive than they do.