[a-dub]

> Given the lack of forecasting signal, the obvious next step then is to seek out external (exogenous) features in the real world that can help prediction models.

this seems right to me. maybe another interesting approach would be a fusion llm+ts model that does multiple-input-single-output with input metadata and causality narrative. so it "thinks" about what data it has and how predictive it may be of the target variable and when something "interesting" occurs it uses the big priors to synthesize a good guess at what it would look like.

so you'd have something like the time series data plus textual narratives of the causality stories as the training data.

[jamilton]

There's a couple "AI forecaster" companies now, Preseen and FutureSearch. My understanding is you give them a query and they research every possible relevant factor and give an estimate of the likelihood of your query, and supposedly they're comparable to if you asked a human to do the same thing, but they're new enough that they're not really proven yet.

[a-dub]

something like that would be interesting. i just get the sense that doing it metadata blind would result in something that learns a somewhat brittle short-time covariance-like structure that doesn't use all available information to achieve best possible generalization and that in the end the benchmark numbers start to reflect some luck with that for predicting non-stationary behavior combined with a lot coming from a pretty good synthesizer that matches previous patterns.

that being the case, it seems the next jump in performance would come from incorporating both metadata and metadata enriched causality and maybe that next jump in performance would be the most interesting jump from a practical system that is useful perspective. (it's more valuable for a system to predict outlier events than it is for it to do an excellent job at synthesizing ordinary behavior)

[thomasikzelf]

I did some energy price forecasting and my neural network learned the morning evening patterns just from historical data, it worked great. Then I tried adding more information to this like wind and solar (and many more). None of these external factors made any difference while being very important for energy prices. It seems like the market already incorporated all the factors in the price (as they should).

The takeaway might be that historic data of market data might often be enough to make a reasonable prediction. Only external data that nobody else has (used) can make your prediction better then the market.

Making a prediction with same accuracy as the market: easy Making a prediction with more accuracy then the market: very hard

[a-dub]

> The takeaway might be that historic data of market data might often be enough to make a reasonable prediction. Only external data that nobody else has (used) can make your prediction better then the market.

that's basically the definition of an efficient market. demand forecasting and insuring against price shifts is the actually useful thing that the commodities futures markets do.

also a fundamental difference from other time series prediction problems. there are all sorts of weird dynamics that go on before one biosignal effects another, or one metric predicts a failure, where a reasonably efficient market price reacts quickly to well known exogenous factors.

[gkbrk]

The best advertisement for these companies would probably be running a public Polymarket account so everyone else can quickly check how accurate they are.