[sgt101]

I think that a Black Swan is when a new factor appears in your domain. In science we are conditioned from the start to create fair tests in controlled experiments. Control is the fundamental of experiment - and statistics are designed to handle experimental data.

In the real world there are often no controls, and complex systems can be driven by an attractor for a very long time before one morning they are not, and every rule that you have is useless (often worse than useless).

Sources of error are not equal; "Black Swan Error" is unusual in that over time it may be that this source is more important than any other source of data in your domain - the strange attractor that drove the creation of your classifier over the last 20 years may never recapture your function and if that's the case your classifier will be literally the most wrong thing you could have!

[ben_w]

That's certainly one type of Black Swan. Taleb's example of that sort of thing being a Turkey predicting they will be fed (because that is what happened every other day of their life) but who is actually slaughtered.

However, it is not the only type. There is also the stock market, which demonstrates major unpredictability every few years, but which can also be approximated the same way between each of the Black Swans. (And they keep being Black Swans because the gap between them is large enough for people to convince themselves that "This time it's different, this time n̵o̵b̵o̵d̵y̵ ̵w̵i̵l̵l̵ ̵h̵a̵v̵e̵ ̵t̵o̵ ̵b̵e̵ ̵n̵a̵i̵l̵e̵d̵ ̵t̵o̵ ̵a̵n̵y̵t̵h̵i̵n̵g̵ growth will be eternal!")

Edit:

Point is, it generalises as how wrong you are in your predictions, and the closer your estimate of your error rate is to your actual error rate, the better your model is.