[thaumasiotes]

The valid way for it to work would be to test the model's ability to predict the future. You make your models, collect data, and then reject models that couldn't accommodate reality.

This isn't done, particularly in climate science, because it's slow and difficult, but "model validation is slow and difficult" doesn't add validity to invalid models.

[mirimir]

Of course it's done! For one thing, you don't need to fit your model to the full historical record. So you can use more recent data to validate models fit to older data. It's like training AI.

But yes, models fit to the full historical record can only be validated in real time.

[thaumasiotes]

> So you can use more recent data to validate models fit to older data. It's like training AI.

This doesn't work in an adversarial context; there's no guarantee that the modeler didn't cheat.

[mirimir]

Hey, science is adversarial. Climate datasets are publicly available. And there are mechanisms to detect and punish cheating.

[thaumasiotes]

Science as practiced is generally not adversarial. Witness the reproduction crisis.

[mirimir]

Well, that's psychology, and calling it "science" is arguably iffy. I've attended a fair number of seminars in physics, biochemistry and molecular biology. In the right hands, a "dumb question" can be a deadly weapon. And if the presenter is a prospective hire, it's often all the more intense.