[dagw]

I think everybody agrees that something went wrong and many people think that something should be done (although I personally think that medias inability to interpret and report on polling errors and uncertainty did greatly exasperate the situation). I just haven't seen any good articles making a strong argument about what went wrong and how to fix it

[pavelrub]

I don’t know, I’m under the impression that people here think that winning with a 20% predicted chance of winning isn’t indicative of anything wrong with the polls, because it’s like predicting the outcome of a die, and therefore nothing should be done at all. This false analogy is precisely what I’m trying to disprove.

I agree that there are currently no strong arguments about what were wrong and how to fix it, but I think it's because it takes time to investigate those things. According to Pew, the American Association for Public Opinion Research has a committee investigating it, and they should release their report in May. It took a 6-months investigation to come up with a report about what went wrong with the Brexit polls, so it will probably take a similar amount of time with this.

[dagw]

You're conflating the polls with the predictive models. The polls never said Trump had a 20% chance of winning because the is not how polls work.

There where a dozen or so models which took the polls as input (some added other inputs as well) and produced a probability of a candidate winning. Some of those models where garbage (Huffington post I had Trump at ~1.5-2%) and some where pretty good (fivethirtyeight had Trump at ~30% and trending upwards). The question about how you should interpret these numbers is a more open one. What does it mean to give numeric probabilities to events which are completely unique and will only occur once (cue discussion of Bayesian vs Frequentist inference here).

Admittedly the question about if the model was right or wrong is difficult to disentangle from the question about bad polling. No model can work correctly if you feed it garbage data. The only criticism you could make is that they should have been even more critical of data they where getting from certain polls than they where.

Now as to was there something wrong with the polls? Obviously. But the interesting question is what went wrong. They where pretty good at forecasting the national popular vote, while at the same time getting certain mid-western swing states dramatically wrong. So there is obviously something in their methodology which seems to works fine when looking at the country but fails when looking at certain states.

But like you I look forward to seeing more detailed investigations coming out in the next few month.