[canjobear]

Has their model made accurate predictions on held-out data? That's all that matters.

[dredmorbius]

For those curious about the term "held out data", it's a modelling / validation method.

https://people.duke.edu/~rnau/three.htm

[jacquesm]

Wrong domain.

[dredmorbius]

How so?

I'm presuming that held out data is used in gradient descent machine learning artificial intelligence, and isn't applicable to systems theory models.

And that other options, including backcasting or parameter adjustment (as was done with different scenarios tested under World3) are.

[onethought]

Ability to predict a past future doesn’t give any strength to the ability to predict the future future.

[betwixthewires]

I'm guessing you're using an analog to "past performance doesn't predict future performance" which is true.

However, seeing as the default likelihood of predicting any future with certainty is zero, a track record of a model of being entirely unable to predict the future at all does very much indicate that such a model will continue to fail, more often than not.

Predicting the future based on what has happened is not reliable, but predicting the future based on models that have predicted the future before should be reliable, if the models themselves are based in reality.

[onethought]

how did everyone’s model of 2019-2021 pan out on any given metric? Even if they successfully predicted that same metric correctly for the last 100 years.

I guess weather/climate models stay somewhat consistent. But they also don’t predict very far forward.

[bryanrasmussen]

Ok so assuming you had a system that you could give it a bunch of inputs and it would predict what the results from those would be 10 years out, and it was correct 60% of the time from past observations, and you had run it on thousands of simulations (this is obviously a hypothetical) is it then your contention that if you gave it inputs from our current inputs and had it predict results ten years from now that those predictions would not have a 60% chance of being correct?

I mean sure, since there is such a thing as an inductive gap in theory, but it seems to me from your statement that you think it to be an insurmountable obstacle in practice as well?

[cryptonector]

It's obvious enough. Conditions change. Systems are dynamic. A model of human society that "works" for the next 10 years may not work for the subsequent 10 years.

[baq]

and yet we model complex stuff with some success.

[onethought]

Like how many computer chips we’ll need during a pandemic. (Sorry couldn’t help myself)