[pedrosorio]

If they were wrong, but “less wrong” than all others, you should pick their model (unless you have an oracle, because the alternative - flipping a coin or “relying on your intuition” is rarely better).

Also, it doesnt make sense to look at a single prediction to evaluate a model.

Out of all the predictions they have made (did you look at individual state predictions?), how many were correct (and how confident were they?) - how many were wrong (and how close to 50% were they?).

That is how you evaluate a model (aka cross entropy)