[learnstats2]

On the other hand, this is a predictive task that has defined outcomes and clear historical data - by my understanding, it is easier than commercial uses of machine learning [at least, easier to measure the effectiveness].

It's also Goldman Sachs and UBS choosing to attach their names to these and stake some reputation on these predictions. If they had hit the bullseye, they would be lauding these results.

[CoryG89]

It may be easier to measure the effectiveness (give a confidence level for the prediction), but just because there is clear historical data and defined outcomes, that does not mean you will be able to predict a particular outcome with any high level of certainty.

For example, imagine a tournament with a large number of participants, where the winner is picked simply by fairly choosing a single random participant.

If I then gave you all the perfect historical data going back decades, you could do statistical analysis and determine that the winner is completely random and therefore the probability of success, for any particular participant, is p~=(1/n), where n is the number of participants. Your confidence in correctly predicting any particular outcome will drop as n rises.

Not everything can be easily predicted just because you have enough data.

[sgt101]

Yup, the worst thing is that if they had got it right it would have been more or less due to pure chance, and it would have led to business flowing their way!