[TFortunato]

Oh that is awesome, and I'll have to check the paper out! That's actually something I've been wondering -- about combining the power of deep learning with the domain expertise of old style expert systems, whether for bootstrapping the neural net, or as a rule base it can consult.

It seems like it would be very powerful to be able to harness Neural Nets for dealing with fuzzy inputs (images, natural language, spoken words), but adding a propositional rule-base they can consult for whatever the actual task is once you have dealt with that input.

On that, if it was learning a rule base, it might also be really helpful with getting insight into what your model is doing, if you could somehow introspect on / view the rules that it learned on a higher level than "when these neurons fire, we do this to the output"

(I woouldn't be surprised of course, to find one day that the DeepMind guys are already on top of it, and come up with a "Neural Warren Abstract Machine" at some point)