[letitleak]

One of the reasons I like crowd prediction is that it seems to be slowly leading to well distributed groups of people who are able to make educated guesses even with one or two conditionals.

While governments have professional analysts, they don't do all that much better than chance even on direct predictions and then what they do share with the public is filtered by their biased interests.

Not only do we not know, for example, if the chaos after an accelerated arab spring was a better outcome from the perspective of maintaining western financial/political interests at a cost of preventing a more gradual transition into stable democracies. But more importantly, we also don't know if government analysts had that hypothesis and they still chose to interfere.

Personally, I don't think western democracies will continue to function with that type of secrecy of knowledge combined with increasing computational modeling/prediction capabilities.

[jbattle]

Is there a data set somewhere that shows concrete predictions made by groups of people and how often they are right/wrong?

[letitleak]

There are research projects/papers on the topic with claims that are all based on Brier scores. I'd say Brier scores are inherently biased by choices in framing/asking questions, but at least measure everyone uniformly/fairly who is given the same questions:

https://en.wikipedia.org/wiki/The_Good_Judgment_Project

For full datasets, I'm not sure if they provide it since not all researchers are good about open data. You could look at open prediction markets which are directly observable but they wont give you any insight into how professional analysts compare with the crowd.