[stingraycharles]

Noob question from me: what’s the difference between accuracy and calibration? A well calibrated market would be more accurate and vice versa, not?

Edit: just found the answer myself: “accuracy measures the percentage of correct predictions out of total predictions, while calibration assesses whether a prediction market's assigned probabilities align with the actual observed frequency of those outcomes”

[kurtis_reed]

Suppose there are 1000 events and 500 will have outcome A and 500 will have outcome B. If you predict a 50% chance of A for every event you'll be perfectly calibrated. On the other hand, if you predict a 90% chance of a certain outcome and you're right for 800 events, you're not perfectly calibrated but you have a lower Brier score (lower is better).

[kqr]

A forecaster can be calibrated but almost only assign probabilities in the 40--60 % range. This is not as ueful as one assiging calibrated probabilities in the full range.

We try to measure the increased usefulness of the latter with proper scoring rules.