[bo1024]

Nice article. One small comment, it's very hard to conclude anything about accuracy over time because the comparisons may not be apples to apples. For example if there used to be lots of questions about if it will rain in Boston and now there are lots of questions about if it will rain in Phoenix, it will look like predictions are getting more accurate, but the questions are just getting easier.

[ddp26]

Author here. Agree, and I wrote in that section "Absolute accuracy is hard to compare across markets on one platform, and across platforms, because different forecasting questions have different difficulties. I addressed this by tracking similar markets on a single platform over time."

Even doing this, it's not apples-to-apples. One thing is, in this article, I filter only to "interesting" markets, so that controls for the % that are "easy" as you describe.

[bo1024]

Thanks for the reply. Yeah, I think all of your filtering and categorizing makes these analyses really nice.