Hacker News new | ask | show | jobs
by nonbel 2352 days ago
Adding more terms sounds like more overfitting.
1 comments

It sounds like that only because scientists reformulate their models in terms that people are familiar with. Actual physicists don't work in terms of the gamma correction factor; they work in tensor fields that don't look anything like conventional arithmetic.

But they can pare all that down to something expressed in terms people are familiar with. And that has the bonus purpose of helping them understand why the familiar terms were familiar: the "correction factor" is small under circumstances we encounter, and only becomes large under circumstances we rarely do.

If that intrigues somebody enough to learn the actual physics, they'll encounter a completely different and more-encompassing formulation which looks not at all like overfitting. One that turns out to be more elegant, in fact, cramming more information into less notation. But it's information nobody needs until they're doing fairly advanced physics, so we're not going to be teaching it in elementary school any time soon.

Don't worry about it, I think you're talking to a NN stuck in a local optima around the term "overfit".
I don't see how what you wrote could possibly address concerns with adding parameters to the model until it explains current data so perfectly that it cannot generalize to future or other data.

This is not a "theory" problem, it has to do with matching the theory to observations.