[Bubbadoo]

Actually, the problem is not of the human masters' understanding, it's the ignorance of the human master. Some models are so involved, no one knows how they'll behave under certain market conditions, as we saw in the perfect storm of 2008 with credit default swaps, CMOs and CBOs. That's the danger, no matter how smart and clever the creator believes they are.

[robbrit]

What I meant was the machine learning systems that have developed in the last decade since the 2008 crash. Many of these are black boxes with thousands to millions of variables, and it's very difficult to understand exactly why a ML algorithm made a decision that it did.

[pgwhalen]

Is it really that we didn’t know how the models behaved in certain market structures, or that we were feeding them the wrong inputs?

[MR4D]

The former. LTCM blowup in the late ‘90’s is a perfect example of a generally correct trade, but the market moving against them long enough to force insolvency.

Apparently, none of the Nobel prize winners thought to model out that particular adverse condition.

Bunch of smart people + one missed market condition = failure (eventually).