[thomasikzelf]

I did some energy price forecasting and my neural network learned the morning evening patterns just from historical data, it worked great. Then I tried adding more information to this like wind and solar (and many more). None of these external factors made any difference while being very important for energy prices. It seems like the market already incorporated all the factors in the price (as they should).

The takeaway might be that historic data of market data might often be enough to make a reasonable prediction. Only external data that nobody else has (used) can make your prediction better then the market.

Making a prediction with same accuracy as the market: easy Making a prediction with more accuracy then the market: very hard

[a-dub]

> The takeaway might be that historic data of market data might often be enough to make a reasonable prediction. Only external data that nobody else has (used) can make your prediction better then the market.

that's basically the definition of an efficient market. demand forecasting and insuring against price shifts is the actually useful thing that the commodities futures markets do.

also a fundamental difference from other time series prediction problems. there are all sorts of weird dynamics that go on before one biosignal effects another, or one metric predicts a failure, where a reasonably efficient market price reacts quickly to well known exogenous factors.