[modeless]

I was similarly disappointed when I read this, but upon further reflection I still like this paper. It is very plausible that both of these problems could be fixed, it would just take a lot more time/power to train, and the resulting system would likely not run in real time making it impossible to test against real humans.

Further advancement in this area will require huge leaps in hardware performance. Luckily in the next few years I expect that the pace of improvement in specialized hardware for neural nets will far outpace Moore's Law.

[gwern]

I'm not nearly that pessimistic. Beating SSBM is well within the capability of a well-tuned A3C, and definitely within the capabilities of a group like DeepMind. More neuromorphic hardware is unnecessary and with current RL methods, they are more CPU-bound than GPU-bound (take a look at the NN they use, it's trivially small; most of the computation goes towards running many SSB games in parallel in order to generate any data to do some small updates on the NN).

I believe they've handicapped themselves, actually, with their shortcuts: the performance of agents is crippled by the inability to see projectiles due to the choice to avoid learning from pixels (which I bet would actually be quite fast, as learning from pixels is not the bottleneck in ALE), and likewise the use of the other RAM features is the path of the Dark Side - allowing immediate quick learning through huge dimensionality reduction, seductively simple, yes, yet poison in the end as the agent is unable to learn all the other things it would've learned (such as projectiles). I suspect that this is why their current implementation is unable to learn to play multiple characters: because it can't see which character it is and what play style it should use.

So I would not be surprised at all to hear in a year or two that human-delay-equivalent agent using raw pixels could beat human champs routinely.

[vladfi1]

The main blocker on using pixels has been getting them from the emulator. I doubt pixels would give a big advantage over RAM features, especially after projectile and stage info is added (there's a PR pending). Captain Falcon (the main character used) doesn't have projectiles anyways.

In fact, RAM features are likely to be much more useful for model-based approaches, which may be important for solving the action-delay problems.

As for multiple characters, the character ID is available to the network. I doubt pixels will be help there either.

[nickpsecurity]

"I'm not nearly that pessimistic."

Me either. Bots for fighting games have always been easier to write or even fake for the button-mashers. This proves nothing. It's just fun. Let's see them get top 2-4% skill & kills at Battlefield 4 with shots hardwired to miss 30-50% of time, playing with weakest carbines w/ suppressors, and on weak teams. If these AI's are so amazing, let's see them similarly use good tactics in open-ended battles to win as I do with a brain injury. I'll even give them training data in form of virtual lead. :)