[fddr]

They are giving probabilities for discrete events, which already captures their level of uncertainty. Probabilities of probabilities (i.e., a probability distribution of a probability) are not very useful concepts.

[subroutine]

It looks like they are simply providing the summary data to their multiple choice survey questions. Still, CI or SEM would not apply (there is no SEM for 80% of forecasters said 'yes'). These graphs are literally just telling you the percent of responders who picked a given answer to the corresponding question.

edit- the longer I browse their website for the exact methodology, the less impressed I am with this group. The "Introducing the Superforecasters" section is so cringe.

https://goodjudgment.com/about/

[alexilliamson]

This would imply that the confidence interval around the coefficient in a logistic regression is not a very useful concept, which I don't think is true.

[fddr]

That it is a little different. There you are estimating a continuous parameter (which happens to be interpretable as a probability) and it makes sense to have a probability distribution over that.

But if you are talking about whether a single discrete events will happen or not, a single number (the probability) already fully captures the uncertainty about it.

[diab0lic]

There is a lot of missing information in their probability of binary events presented. Presumably they polled N forecasters and are presenting an x/N prediction. The fact that each forecaster is estimating in a continuous space and then binarizing their result means that a lot of information has been lost.

To look at an extreme example… were all the “yes” votes 95%+ certain and the “no” votes just under the line 49%? Or was it more like a bunch of no votes at 49% and a bunch of yes votes at 51%?

Binarizing forecasts necessarily discards information. Aggregating a bunch of binary predictions into a percentage does no recapture said information, unfortunately.