[graemep]

How do you test the predictions of climate science models? I have never found a satisfactory explanation.

There is insufficient data for backtesting models of that complexity.

Arrhenius's original model have help up pretty well, but it is also simple.

Some models from decades ago including some that were very widely publicised in the 80s/90s have been badly wrong.

[jeffbee]

> I have never found a satisfactory explanation.

Translation: I have never looked for an explanation.

"""Time series of global average temperature anomalies show excellent agreement between observations and model simulations (Figure 1b)."""

Randel, W. J., Polvani, L., Wu, F., Kinnison, D. E., Zou, C.-Z., & Mears, C. (2017). Troposphere-stratosphere temperature trends derived from satellite data compared with ensemble simulations from WACCM. Journal of Geophysical Research: Atmospheres, 122, 9651–9667, https://doi.org/10.1002/2017JD027158

Among many, many others in the literature. Climate models going clear back to 1960 have been right on the money, with the primary source of error being predictions about how much stupid shit humans would do.

[graemep]

Nice snide comment to start with. I have looked. What I have found does not match what the models need. Most things aimed at the general public are simplistic. I assume the real answers require access to journals.

Are global average temperatures enough to provide data for all the temperature variables in models? Surely only sufficient for a model that predicted only global average temperatures.

Some models may have been right, but some were badly wrong Here is an example of one that got media attntion that is clearly badly wrong.

https://web.archive.org/web/20100212233059/http://www.indepe...

[jeffbee]

But was it snide enough? I want people who try to poison the discourse with comments like yours to know that they have no place in civil society and are being corrected.

[lucb1e]

> What I have found does not match what the models need.

I don't understand this part. Do you mean there is a mismatch between the input parameters to these models and what data you've found to be available?

As for access to journals, yeah, sci-hub is also where I turn to but it's getting less and less reliable. I should make acquaintances with someone at an allowlisted IP address at some point...

> Here is an example of one [model] that got media attention that is clearly badly wrong [URL]

Article is from the year 2000, for context. Parts that have data in them (past or future):

> > in the south of England, for instance, from 1970 to 1995 snow and sleet fell for an average of 3.7 days, while from 1988 to 1995 the average was 0.7 days. [...] Average temperatures in Britain were nearly 0.6°C higher in the Nineties than in 1960-90, and it is estimated that they will increase by 0.2C every decade over the coming century. [...] within a few years winter snowfall will become "a very rare and exciting event" [...] a wide range of research indicates that pests and plant diseases, usually killed back by sharp frosts, are likely to flourish. [...] Heavy snow will return occasionally, says Dr Viner, but when it does we will be unprepared. "We're really going to get caught out. Snow will probably cause chaos in 20 years time," he said.

Hmm so I'm not seeing the mention of a particular model or study here, just observations of past snowfall and vague mentions (by figures of authority on the matter, to be fair) of "within a few years", "occasionally", and "probably in 20 years". The only real predicted data here is the 0.2°C warming per decade until 2100, but that makes little sense: the day we stop emitting greenhouse gases, the trend will start to slow until it stabilizes (afaik that doesn't take more than one or maybe two decades). It seems foreseeable that we can reduce fossil fuels by at least 50% within that coming century, so how could it keep warming at that rate?

So that seems immediately suspect to me and I wouldn't trust it, but let's say that a casual reader takes it at face value, and for the next handful of decades (until ~today) it should have come true at least. Let's see what actually happened (I'm typing this as I find out, I don't know the answer at the time of writing). The sentence mentions Britain specifically, so that's the right island of the two big ones in the UK afaik, so I searched "britain historic temperatures", found as second result some weather office page (very findable for once! nice), picked a station at random near the middle of the country (at, what seemed to me, an average distance from the coast, to have something potentially representative), and got to this data: https://www.metoffice.gov.uk/pub/data/weather/uk/climate/sta... It breaks down temperatures per month from 1957 through 2022. I copied the data lines into this script:

    import sys, re
    decs = {}  # decades
    for line in sys.stdin:
        y, m, tmax, tmin, *rest = re.sub(' +', ' ', line.strip()).split(' ')
        dec = int(int(y) / 10) * 10  #find decade a year belongs to, e.g. convert "2003" to "2000"
        if dec not in decs:
            decs[dec] = []
        decs[dec].append(float(tmax))

    for dec in decs:
        # print the decade and the sum of all values divided by how many values are known (i.e., print the average value) to two decimals
        print(str(dec) + 's', round(sum(decs[dec]) / len(decs[dec]), 2))

And got these results (manually added the 'incomplete' markers for decades where we don't have data from all years):

    1950s 13.53* incomplete
    1960s 12.72
    1970s 13.17
    1980s 13.15
    1990s 13.68
    2000s 13.95
    2010s 14.04
    2020s 14.65* incomplete

So the article said the 90s should be 0.6°C warmer than 60s-80s: (12.72+13.17+13.15)/3 = 13.01; 90s = 13.68; difference = 0.67°C. Okay looks like we've got the right data here when using the daily maximums (though it's still only one weather station; could be coincidence). Then the 2000s were 0.27°C warmer and the 2010s another 0.09°C (average: 0.18 per decade).

Doing the same but with tmin instead of tmax (replace that word in the line that contains 'append'):

    1950s 5.17* incomplete
    1960s 4.91
    1970s 5.0
    1980s 4.95
    1990s 5.5
    2000s 5.85
    2010s 5.74
    2020s 6.2* incomplete

60s-80s average is 4.95°C; 90s is 5.5; difference is 0.55°C. Again, sanity check: it matches what they claimed historic data was like. The 2000s are 0.35°C warmer, that's quite a bit more. The 2010s are cooler, showing -0.11°C warming compared to the 2000s, so that one is clearly out of line. Taken together, you're looking at 0.12°C warming per decade on the daily lows.

Neither is precisely correct, but about +0.15°C (looking at both tmax and tmin) also not super far off for a one-decimal-precision figure to begin with and one random weather station (I picked only one, I haven't cherry picked it).

== Conclusion ==

- The temperature prediction I would give a tentative pass (checking more data would further strengthen or weaken that).

- The claim about heavy snow being 'unexpected' 20 years from then (i.e., three years ago), I dunno, traffic is definitely chaos in germany when any amount of snow shows up, but that's not new as far as I know. What constitutes 'heavy snow' anyway? I certainly don't expect heavy snow any given year, and those historic UK figures look lower than what we get today (which makes sense for an island: I'm about as far inland as GB is wide). Hard to say for me whether that came true. What is your take on it?

- The predicted pests and plant diseases did not have a date on it. That may have been a general statement about warming weather. Not sure where I'd find data on pest and plant disease occurrence, especially normalized for use of insecticides or such.

- The only part that I find wrong is the subjective statement `winter snowfall will become "a very rare and exciting event"`, based on my experience in Germany. It's certainly not the norm for an average winter's week, but "very rare" also sounds too strong. Then again, GB's historic figures seemed lower so who knows, but I'm out of energy (I'm literally over an hour into this comment, I hope you find it useful at least!) to find further data on snowfall in GB/DE or do a survey of GB's reaction to snowfall.

Conclusion's conclusion: the only thing for which I have data was close enough, the rest was vague to begin with.

What part of the article did you find 'clearly badly wrong'?

[lucb1e]

> (I'm literally over an hour into this comment, I hope you find it useful at least!)

aaaaaaand ghosted...

I should really start to err more on the "fuck this" side rather than dilligently verifying posted information and trying to work toward a common understanding

[lucb1e]

Answering that to the detail that I feel it requires to prove beyond doubt, would likely take about as much time as doing a meta-analysis on the topic. Here's some starting points:

https://www.climate.gov/news-features/climate-qa/what-eviden...

> We know this warming is largely caused by human activities because the key role that carbon dioxide plays in maintaining Earth’s natural greenhouse effect has been understood since the mid-1800s. Unless it is offset by some equally large cooling influence, more atmospheric carbon dioxide will lead to warmer surface temperatures. Since 1800, the amount of carbon dioxide in the atmosphere has increased from about 280 parts per million to 410 ppm in 2019. We know from both its rapid increase and its isotopic “fingerprint” that the source of this new carbon dioxide is fossil fuels, and not natural sources like forest fires, volcanoes, or outgassing from the ocean. [...] no other known climate influences have changed enough to account for the observed warming trend

If you're wondering how to establish causality in the first place, when we can't do experiments (we could hardly add and remove a significant amount of emissions at will to see what effect it has), this article provides two methods: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7081045/ The problem is not unique to climate science. Quoting the "background" section:

> In clinical medical research, causality is demonstrated by randomized controlled trials (RCTs). Often, however, an RCT cannot be conducted for ethical reasons, and sometimes for practical reasons as well. In such cases, knowledge can be derived from an observational study instead. In this article, we present two methods that have not been widely used in medical research to date.

Beyond global warming, much easier to study is the effect that air pollution has on humans. Stopping coal power plants and combustion engine vehicles in cities would also extend millions of people's healthy years of life every year. From https://en.wikipedia.org/wiki/Air_pollution

> Outdoor air pollution attributable to fossil fuel use alone causes ~3.61 million deaths annually, making it one of the top contributors to human death

The solutions we have today (using wind, water, sun, and fission) that reduce warming and improve health are not perfect, but my understanding is that they are better

[graemep]

How does that relate to what I said?

I am not disputing that adding CO2 to the atmosphere will net cause warming. I am asking for the methodology used to test climate models. The same with the fact that pollution is bad: nothing to do with the testing of climate models.

The analogy with medicines where controlled trails are not possible is flawed. You can observe large numbers of people. We can only observe one planet.

From the replies and the down votes, its seems people read even asking questions about climate models as a claim that climate change is not happening, or human beings are not a major cause, or something similar.

[lucb1e]

> I am asking for the methodology used to test climate models.

I do not have specific sources about that myself, perhaps someone else

> is flawed. You can observe large numbers of people. We can only observe one planet.

Good point, I hadn't considered that. Still, one can observe a lot of states over time: Earth with an average global temperature of x°C where the CO2=x, sun_state=y, number_of_penguins=z, etc., then the same set of parameters for another temperature. Check which one turns out to always be followed by a rise in temperature the next month/year/decade. (If the number of penguins turns out to be highly correlated, remember you read it here first! :P yes I'm joking about correlation vs. causation which, based on what you wrote already, I'm sure you know about.)

Whether the methods from the paper I found earlier can work 1:1 with this other approach, I'd have to check (or find what other methods are/were being used in climate science specifically), but that's how this must be done because we indeed have barely any data on other earthlike planets

> its seems people read even asking questions about climate models as a claim that climate change is not happening, or human beings are not a major cause, or something similar.

Yes, I've noticed that as well. There are some downvoted comments of mine (also on reddit) for the same reason. It's a very fine line to walk, making it clear that you're obviously understanding of reality but also wondering about some detail aspect of climate change

When there's some doubt and it could be a legitimate question, I always try to find the time to comment without downvoting because, that way, we keep in conversation and other people might also get more insight, but alas I don't always have time and it's common that I see posts (even on a place like stackoverflow where posts are typically rather objective) downvoted without commenting the reason