[KKKKkkkk1]

My biggest issue is when people say that it's a probabilistic model, and therefore it wasn't wrong in 2016 because 28% chance of winning is pretty high and you don't get probabilities. Well, guess what, this kind of model that provides a probabilistic estimate on a future event that cannot be repeated cannot be validated or falsified. It's basically junk science (if it has any aspirations of being scientific).

[Marazan]

If I tell you an unweighted 6 sided die only has a 1 in 6 chance of coming up 6 and we roll it once and it comes up a 6 then that is not junk science.

[bena]

Yeah, but what if we could never roll that particular die again?

I think that's what he's talking about.

[mac01021]

This is just Frequentism vs Bayesianism, right?

https://en.wikipedia.org/wiki/Probability_interpretations

[NhanH]

You treat the forecaster as the dice, not a particular forecast from them. You can measure their forecasts with reality.

[bena]

Yes, I'm not defending the original guy, I'm just stating what I believe his argument to be.

He's basically saying he doesn't believe the die is fair/unweighted.

So stating the odds of a fair die is kind of immaterial to his point. We need to demonstrate to the guy that the die is fair.

Someone else posted a link (I think) of 538 going over on how accurate they were. Whether their odds bore out. Here it is: https://projects.fivethirtyeight.com/checking-our-work/

Basically what they did was bucket every prediction by odds. If they predicted 70/30, it went in that bucket. And they're "right" at about the rate of their predictions. In other words, for every 70/30 prediction they made, the people/teams with 70% chance to win, won about 70% of the time.

That shows that 538 in this case is pretty decent at calculating odds.

[bart_spoon]

Yes, which is why it’s foolish to talk about the outcome of that single race. You are right that there will never be another 2016 election between Clinton and Trump. However, that is only one of hundreds of forecasts made by 538 across multiple election cycles, so we can see how often their probabilistic outcomes align with actual outcomes. The fact that their track record is fairly good across all of these is evidence suggesting that Trump’s win may be more likely to be the realization of a less likely outcome predicted by their model, rather than a fundamental problem of their model itself.

[bena]

Like I told the other guy, I'm not defending his position, I'm stating it in different terms. And stating why pointing out the odds of a fair die isn't a good counter argument.

Grandposter doesn't believe the die is fair. That's a different argument than the guy I responded to made.

[bigbubba]

Sounds more like gambling than science; you're making a one in six bet that you can dupe me. More seriously, I would not accept your claim with evidence as weak as that. Roll the dice a few more times and then I'll credit your claim. Rolling the dice only once when you could just as easily roll them a dozen times is junk science.

Incidentally, this trick is something magicians sometimes do. Sometimes when a trick has gone wrong they'll make a wild guess. If they're right, the audience is impressed. If they're wrong, they'll brush it aside with some joke and the audience won't notice/mind much. This works for things card guessing tricks and puedo-psychic/cold reading stuff.

[stjohnswarts]

So all statistics and probabilities are junk science because they can't predict the future 100% of the time? Surely you can't be asserting that...

[apsec112]

You can check their forecasts on Senate and House races, where we have a lot more data points.

https://projects.fivethirtyeight.com/checking-our-work/

[civilized]

Or... if you think for a second about what probabilities are supposed to mean, there is an obvious way to check if a probabilistic forecaster is accurate https://projects.fivethirtyeight.com/checking-our-work/

[HeavenFox]

This is basically the frequentist vs bayesian debate. Looks like you are firmly in the former camp :)

[cwhiz]

No one is purporting election forecasting to be scientific.

[bigbubba]

Some of your sibling comments seem to suggest otherwise.

[cwhiz]

Well, HN commenters don’t speak for the people doing election forecasting. I think Silver would describe it as educated guesswork.

[bigbubba]

I suppose you meant that election forecasters don't claim election forecasting is a science, not 'no one.' I don't mean to quibble, this meaning was not at all clear in your original comment, nor did I expect that meaning from the context of the comment you responded to.