[Zolde]

One should ask economists what a recession is, not how to predict one. Good modelers do not necessarily need (or want) to know what they are predicting and still beat "domain experts".

Authority without clear track-record is a net negative to getting good results. It is better to stick to anonymity, and only let the track-record do the talking/weighting. Without a clear track-record it does not even matter if the prediction-maker has skin in the game. If you do have skin in the game, there is no reason to sell your hide cheaply, or even give it away. You instead take the profit others say does and can not exist beyond "luck": If you can't even beat a random walk, you have no business evaluating the limitations of predictive modeling.

The big consultancy companies making bold predictions don't even need to be right. Customers read the predictions these consultancy companies peddle, because these customers are not bold enough to make their own predictions. And nobody ever got fired for buying the predictions from big consultancy companies and incorporating them into a business strategy.

[hef19898]

Consultancies predicting something isn't forecasting, it is marketing.

And there or only a rare few thing I disagree more stongly with the statement, that good modellers / data scientist / whatever only need knowledge about how to model stuff to beat domain experts. It takes domain experts to judge whether or not a model correct, to identify the known and unknown unknowns and limitations of these models. Claiming otherwise is deeply arrogant, and it ended in disaster everytime I saw it tried. Good modellers need enough domain knowledge to properly work with, and understand, domain experts. And domain experts need sufficient knowledge about modelling to do the same. Both need the willingness to do so. And every modeller needs to accept that reality beats models, always.

[Zolde]

"Every time I fire a linguist, the performance of the speech recognizer goes up."

> It takes domain experts to judge whether or not a model correct, to identify the known and unknown unknowns and limitations of these models.

Arguably true, but I still claim the domain expert test-performance is below that of a modeling expert. No knowledge/preconceptions: Try it all, let evaluation decide. Expert domain knowledge/preconceptions: This can't possibly work!

Domain experts need to focus on decision science (what policies to build on top of model output). Data scientists need to focus on providing model output to make the most accurate/informed decisions downstream.

[hef19898]

I'll be blunt: everytime I saw people try model something they don't understand, it boiled down to throwing stuff at the wall and see what sticks. Very best case, whatever stuck solved one special case without people realizing it was a speciap case.

Worst case, the stuff sticking was sheer luck, could have, and quite often was, identified prior of trying by domain experts, no lessons were drawn from the excercise and the resulting models were ignored by everyone except the modellers.

[notahacker]

> One should ask economists what a recession is, not how to predict one.

Most economists would agree. It's everyone else that says "well if you know so much about how shocks and policy changes cause recessions, why can't you tell me if there will be a recession in $country in Q2 2025?". And in economics, "skin in the game" means policy responses to avoid dire forecast outcomes (or lack of them when nobody expect oil prices to change or a major bank to collapse).

There's no shortage of opportunity to make money by beating everyone else at the prediction game, but the funds that have consistently profited from spotting the recessions ahead of everyone else don't exist any more than the always-right public expert forecasters.