[highd]

If you're comparing deltas (i.e. x_{n+x} - x_{n+x-1}) that might be sufficient - otherwise it's hard to tell if you're just capturing that x_{n+1} is close to x_{n}. The primary risk would be that you're putting strong structure on the datasets you're testing with, so you could be mislead. Ie what if you have:

  y = sin(t) if 0<t<100Pi

  = sin(2t) if 100Pi<t<200Pi

  = sin(3t) if 200Pi<t<300Pi

Then you could imagine that with simply backtesting the model in front of where you're training you could run into issues - each train iteration might fix a constant frequency in the network and then it looks like it works great over each iteration, but you've never learned how to determine each frequency on-the-fly. If that happens with random backtesting from the dataset the backtesting would show that only 1/3 of the test set is fitting.

The gold standard is always a well-partitioned dataset. And if you're going to hold a meeting describing your results, or deploy a product, it's really important that the results stand up to these sorts of questions.