[feniv]

Rather than waiting for data from future games, you should backtest on old data and see how it performs (http://en.wikipedia.org/wiki/Backtesting).

You might also be interested in checking out the book The Signal and The Noise by Nate Silver (He runs the FiveThirtyEight political/data blog that notably predicted last year's election results with great accuracy.)

[pondababa]

More relevant--before he did statistical analysis of elections, he was a leading figure in sabermetrics, the statistical analysis of baseball. His program is still one of the best.

[icelancer]

And before that, he was an excellent online gambler. Or was it during that time? ;)

PECOTA is alright, but not really his anymore. cwyers took it over at BPro. It's also tough to say if it's one of the best for any number of statistically boring reasons (read Phil Birnbaum's blog for more info).

[jaredmck]

Yeah, PECOTA isn't one of the best at all, it just has ranges for variance, which is an imprecise metric as no one knows what he's saying a percentile is under the model. What is luck and what is skill?

Birnbaum is one of the best bloggers on advanced stuff. Tango's blog is the best period as anyone good participating in the "open source" movement so to speak is there pr shows up when they write good stuff. Until you're aware of the work to date, you will be spinning in circles with awful biased errors.

[zsch]

Excellent idea. And it will provide a much faster way to gauge its accuracy as I adjust the script to accomodate more areas of information.

I actually just started that book and so far so excellent.