[rcfox]

I assume it would be like finding the interpolating polynomial of a bunch of points in a row. (Like this: . . . . . .) The resulting polynomial would go through all of the points, but have very large peaks/valleys in between.

Essentially, you'd get a system that passed all of your tests, but produced garbage for anything not covered by the tests.

[eric-hu]

Your post got me excited because it sounds like you're describing Runge's Phenomenon ( http://en.wikipedia.org/wiki/Runge%27s_phenomenon ).

The wikipedia article doesn't have the best illustration, but I think the inherent idea is really wise: in trying too hard to meet your initial constraints, you can come up with a solution that's only useful at those constraint points.

[rcfox]

I'd never heard of Runge's phenomenon, but it does indeed sound like I was describing it. Thanks for giving me a name for it.

[eric-hu]

Interesting, I thought that was exactly what you were talking about :)

I learned about it in numerical analysis. It illustrates the downside of trying to be too precise--you can make a polynomial that will go through an arbitrary number of data points.

As the number of points increases, your function will look less like a line and more like a magnitude 9 earthquake on a seismograph. The function will pass through all the points used to define it. However, it'll be useless when predicting the original data's behavior, as it changes too quickly on small input.

Instead, mathematicians find more useful functions by relaxing the conditions so that the model function only has to come 'near' the data points.

[peterhunt]

That's true, but I think this problem will eventually be tractable. We have tools like QuickCheck that can generate randomized test cases and ensure certain properties are maintained. While this probably isn't enough today, I could imagine this becoming a good starting point to improve this technology.