[nrmn]

Yes, it feels like we have squeezed most of the performance out of current algorithms and architectures. OpenAI and deepmind have thrown tremendous compute against the problem with little overall progress (overall, alpha go is special). There was a big improvement in performance by bringing in function approximators in the form of deep networks. Which as you said can scale upwards nicely with more data and compute. In my opinion as an academic in the deep RL, it feels like we are missing some fundamental pieces to get another leap forward. I am uncertain what exactly the solution is but any improvement in areas like sample efficiency, stability, or task transfer could be quite significant. Personally I’m quite excited about the vein of learning to learn.

[an_opabinia]

> alpha go is special

The VC community is in denial about how much Go resembled a problem purpose built to be solved by deep neural networks.

[TchoBeer]

Are you suggesting that Go literally was purpose built for this?

[rwallace]

There is a sense in which it was: out of all the games that have ever been designed, or that it would be logically possible to design, humans selected Go as one of the relatively few to receive sustained attention, in part because it is particularly well suited to the deep neural network that is the visual cortex. So it is not a coincidence that it is also well suited to artificial deep neural networks.

[an_opabinia]

It’s one of the few interesting games out there whose rules can be neatly represented as algebra on binary matrices and still make sense.