[llamataboot]

Simple math applied to a large enough set of predictions will let you know whether or not your probabilities are accurate. It is the same as playing poker - every hand you are essentially betting on probabilities multiple times (okay right now I thing I have a 50% chance of winning this hand and the pot is X and I have to put in Y to keep playing, I'm going to continue). In the short term it is very hard to know whether or not you are putting money in when you "should" because of variance. In the long term, the variance disappears and you can see that, for example, you are making an estimated 2 bets/100 hands you play which means you are "more correct" than the people you are playing against at estimating the odds. A little more complicated than that of course because you are also using bets as weapons to drive better hands out through bluffing and semi-bluffing, etc but the general point holds that a large enough sample size of actual results versus compared results means that your probabilities will hold or they won't

[joosters]

Works great for poker, where you play lots of hands. Works less great for the situation in the article. Just how many North Korean missile launches do you need a person to predict before you know they are lucky or not?

[llamataboot]

Doesn't matter, as long as they predict a large enough sample of events. Just requires an assumption that they will be as accurate on every type of event, which probably isn't true, but close enough for statistics