| The predictions were not so bad. At least one of the favourites won in the end. GS had France winning with 11.3% probability, second to Brazil with 18.5%. UBS was less fortunate, they had Germany (24%), Brazil (19.8%), Spain (16.1%) and England (8.5%) before France (7.3%). I compared the logloss for their predictions with the "uniform" benchmark (giving each team 1/32 probability of winning, 1/16 probability of getting to the finals, etc) and the results are the following (if I transcribed the data properly): Getting to second round: GS: 0.495
UBS: 0.495
bench: 0.693 Getting to quarter-finals: GS: 0.463
UBS: 0.459
bench: 0.562 Getting to semi-finals: GS: 0.310
UBS: 0.327
bench: 0.377 Getting to final: GS: 0.231
UBS: 0.269
bench: 0.234 World-cap winner: GS: 0.097
UBS: 0.113
bench: 0.139 The performance of the models was ok until Croatia got to the finals. This hurt specially UBS, who predicted less than 0.9% probability of such an event (compared to 2.1% in Goldman's model). Edit: these would have been the "best case" scores (if the high-probabilty teams had classified to each round, ignoring that this may be impossible due to the structure of the tournament): GS: 0.432 0.302 0.220 0.141 0.079 UBS: 0.365 0.251 0.176 0.111 0.070 UBS could potentially achive lower logloss metrics because it had more extreme predictions. |