[gus_massa]

Make them play 10 games at the same time. (Perhaps in different shards/worlds to avoid cooperation between the different “personalities”.)

[mrThe]

Nice idea. But in this case, i need to verify, that this is the same robot, because robot can do nothing, or play like a noob in first 9 games, but last game will be played by user through UI.

And you can't verify this, because there is different "worlds" and robot can be not so good, for solve some game problem in some random world.

[gus_massa]

One possibility is to drop the best and the worst results, like in a diving contest: http://en.wikipedia.org/wiki/Diving#Scoring_the_dive

> To reduce the subjectivity of scoring in major meets, panels of five or seven judges are assembled. If five judges then the highest and lowest scores are discarded and the middle three are summed ...

Also, make them play 100 games, and drop 10 of them.

This may work if it is a fast paced game. If it is slow paced with a complex strategy, then a good player may play like in a chess simultaneous exhibition http://en.wikipedia.org/wiki/Simultaneous_exhibition