| There is no strong theoretical foundation for very complex and/or deep models. There are ways to gauge results, but right now, the media is right in that there are risks. The pneumonia project referenced in the article had Caruana voting against implementing the most accurate model: A neural net. Instead they went with a way less accurate logistic regression model, one they could safely implement in production, inspect, explain, and defend to the doctors. Nobody does quite know why neural networks work so well. There is a Nobel Prize waiting there for someone or some team to solve this with mathematical (or physics) rigor. Nodes in neural network layers can represent multiple features, or share feature representations. Do we know if a neural net (and which part) is targeting skin color, or acne? Do we know that credit risk models are targeting sex, even though we left out this feature (it may infer this from other features)?. Depending on the application, this is important to know for certain. Reverse engineering sure does work, but can we fully find out the source code from a program, just by fuzzing inputs and looking at outputs? Or are we only looking at (perhaps a small part of) its behavior? > You will run into trouble if you don't know what you're doing Likewise: You will run into trouble, if you don't know for sure what your models are doing. http://blogs.wsj.com/digits/2015/07/01/google-mistakenly-tag... > as per common sense https://en.wikipedia.org/wiki/Commonsense_reasoning#Commonse... |