[nilkn]

What are your thoughts on newer recurrent architectures like the DNC (or its predecessor, the neural Turing machine)? While the demonstrated results with DNCs so far are pretty limited, it seems that they embody a push towards allowing a neural network to actually "think" over multiple steps: storing complex information, formulating a plan, and acting on that plan.

[daddyo]

Yes. I think these architectures are very exciting and a step in the "right" direction. Eventually we will want to move from rote memorization and pattern matching to more challenging aspects of intelligence.

https://arxiv.org/abs/1601.01705v4 (Learning to Compose Neural Networks for Question Answering) comes close to breaking this barrier.