[jdoliner]

I don't know about the framing of "giving up." But I think anyone who's been following election models since the original 538 in 2008 has probably gotten the feeling that they have less alpha in them than they did back then. I think there's some obvious reasons for this that the forecasters would probably agree with.

The biggest one seems to be a case of Goodhart's Law, leading to herding. Pollsters care a lot now about what their rating is in forecasting models, so they're reluctant to publish outlier results, those outlier results are very valuable for the models but are likely to get a pollster punished in the ratings next cycle.

Lots of changes to polling methods have been made due to polls underestimating Trump. Polls have become like mini models unto themselves. Due to their inability to poll a representative slice of the population they try to correct by adjusting their results to compensate for the difference between who they've polled and the likely makeup of the electorate. This makes sense in theory, but of course introduces a whole bunch of new variables that need to be tuned correctly.

On top of all this is the fact that the process is very high stakes and emotional with pollsters and modellers alike bringing their own political biases and only being able to resist pressure from political factions so much.

The analogy I kept coming back to watching election models during this last cycle was that it looked like an ML model that didn't have the data it needed to make good predictions and so was making the safest prediction it could make given what it did have. Basically getting stuck in this local minima at 50-50 that was least likely to be off by a lot.

[rachofsunshine]

Even if polling had been exactly right, you wouldn't have been that confident in the outcome.

In my unsophisticated toy model, plugging in the exact actual result as the polling average (but not telling it how the actual vote went) spits out 66% R-34% D. Clearly one side favored, but hardly a guarantee. Because the result was close, and even highly accurate data in a close result yields an uncertain forecast.

Remember that asteroid a month ago? We knew what its position would be seven years in the future with a precision of a few hours. But because the position was very close to an impact, even that high precision was not enough to rule out an impact.

[pessimizer]

> Due to their inability to poll a representative slice of the population they try to correct by adjusting their results to compensate for the difference between who they've polled and the likely makeup of the electorate.

This is where polling becomes race science.

> This makes sense in theory

It does not make sense in theory. It is a necessity for the profession, but all justifications for it are specious. Polls only get it right when everybody is getting it right. What they offer is false precision and justification for the current narratives.

It's similar to AI in that way. It's also similar to the mythical prediction markets that polls have been compared to lately, the "mythical" here meaning with no insiders involved. On issues where there are no real insiders, like close elections, the prediction markets are simply a lagging indicator of what pundits said in the paper this morning. That goofy Iowa poll swung them so hard that I thought Seltzer should have been investigated for whatever the prediction market equivalent to securities fraud is.

It might be more accurate in light of the OP to say that polls get it right when everybody is getting it right, and when everybody isn't sure what's going to happen, polling accuracy is around 50/50.

The best book to read about polling is The Full Facts Book of Cold Reading by Ian Rowland. It also tells you how to write defenses like this, which are part of the con.

-----

edit:

Section headings from "The Win-Win Game" from TFFBoCR, which teaches 10+1 ways how to make failures seem like successes:

> 1. Persist, wonder and let it linger.

> (Phase A: The psychic persists with the official statement and tries to encourage at least partial agreement. B: she acts puzzled, and invites the client to share the blame for the 'discrepancy.' C: she leaves the discrepancy unresolved, in case the client finds a match later on.)

> 2. I am right, but you have forgotten.

> 3. I am right but you do not know.

> 4. I am right but nobody knows.

> 5. I am right, but it's embarrassing.

> 6. I am wrong now, but I will be right soon.

> 7. I am wrong, but it doesn't matter.

> 8. I am wrong in fact, but right emotionally.

> 9. I am wrong in fact, but right within [the] system.

> 10. Wrong small print, right headline.

> [+1]. Accept, apologise, and move on.

> (In this way the psychic cuts her losses and moves on. She leaves the problem behind, where it will be quickly forgotten, and at the same time she comes across as extremely honest.)

Very much worth reading for entrepreneurs looking for investment or any other confidence men. Rowland even tried to brand it a few years ago in Cold Reading For Business as the "CRFB" system.

[genewitch]

okay, now change "election prediction models" to "Climate models" and see if you feel like downvoting me merely for pointing out the (slight?) hypocrisy in "excusing" every other model we humans use for being "inaccurate" or "not having the full details" or the "whole slice of"...

when none of the models tend to agree... and the IPCC literature published however often they do it is hung upon the framework of models.

[sojournerc]

All models are wrong, but some are useful.

Climate modeling is way messier than the media portrays, yet even optimistic models show drastic change.

I'm not in the catastrophy camp, but it's worth preparing for climate change regardless of origin. It's good for humanity to be resilient to a hostile planet.

[milesrout]

Yeah my view is a little trite but... "we cleaned the air and closed the ozone hole and reduced our dependency on oil from a small number of OPEC countries all for nothing?".

I support the climate change mitigation and adaptation moves that would be nice anyway (many of the most important ones) and would prefer alternatives to things like turning all the arable land into pinus radiata plantations to generate ETS credits or voluntarily paying "climate fines" to shadowy international organisations if we don't hit certain "targets".

[johngladtj]

That's a great of sidesteping the massive costs of doing all those things

[wizzwizz4]

There are massive costs of not doing those things – and, as far as I can tell, those costs are greater. Coal is expensive and irradiates the atmosphere. Oil drilling causes spills, fracking causes earthquakes, and oil dependency has been a major driver of war for the past century.

[gonzobonzo]

> Coal is expensive and irradiates the atmosphere. Oil drilling causes spills, fracking causes earthquakes, and oil dependency has been a major driver of war for the past century.

Fracking lowers emissions. As far as I've been able to tell, earthquakes aren't a major problem with it.

This is where the activists end up showing their true colors, and why many people have grown to distrust them over time. On the one hand, they claim Global Warming is going to bring us to the brink of human extinction unless we act rapidly. On the other, they're trying to stop things that will lower emissions if it doesn't follow their preferred course of action.

[nextts]

Also the best model (current reality) is showing climate change!

[genewitch]

A++ would upvote again