| A model that takes in 100 bits of specification simply can not correctly describe a process that has 10,000 bit's worth of degrees of freedom. If I'm correct, though, the OP is talking about creating a model with 100 bits of specification, and then creating a model of that model and trying to train those 100 bits, which seems like it should be a more tractable problem. To me it sounds more like he's just rediscovered the fact that when you try to set a model's parameters based on a limited set of observations (he generated 3 years worth of data from his model, then trained parameters based on that data), there's a lot of uncertainty left over, and you won't necessarily get the right model. This is quite obvious - if your observations only cover a limited portion of phase space, then you shouldn't be surprised that in a complex enough model multiple parameterizations will fit the observations equally well. You just didn't have enough freaking data to distinguish between the models! In all branches of science, we deal with this problem, and the solution is that you try to find the simplest possible model that accurately explains your data (or, as is happening in physics right now, you try to enumerate the next level of theories that reproduce current data so that you can figure out which experiments you'll need to run to distinguish between them). So this has doesn't hint at any sort of fundamental flaw with modeling in general (and yeegads, it has even less to do with finance...) - it's just that he didn't have enough data to infer a proper parameterization. Don't build complex models and expect to train them on small datasets... |