[backpropaganda]

Your example about the debate of BN usage demonstrates that it is possible to look inside a deep network and debate. That we don't know the answers doesn't mean the answers don't exist, or are impossible to find, which is what the term "black-box" suggests.

Of course, more research in tools for model interpretation would be awesome, and my own lab has done a lot towards it, and this remains an important topic. More is desired, but what we have right now is pretty good too, and is not at all inferior to old-school methods, esp. considering the performance.

[lightcatcher]

I'd argue that a neural net is "black-box" in the sense that nobody really can give a coherent answer to "what happens if I perturb/double/negate this parameter" where the parameter might be deep in some weight matrix. Maybe this isn't a useful question because of the distributed representations within neural nets, but it is at least an answerable question for other models.

Do you know of any work on interpreting neural nets that are being used for non-image tasks?