[feral]

>The fact that a totally generic AI system with zero knowledge of the game can learn on the fly and beat the hand-tuned AI is pretty impressive.

Yes; but, like, Samuel's learning checker playing programs in the 50s and 60s did that.

>They were training online during gameplay, so each round started fresh with an untrained system.

I'm not really sure how that is considered training. Like Tim_Benham's comment, on a first read of the paper, I would be concerned that the AI is beating to inbuilt AI due to what could be termed 'save-game cheating'. And there's still the issue of whether it is overspecialised to just exploit the inbuilt AI in some way.