[ben_w]

Black Swans are the error rate of your predictions (the real error rather than your prediction of your error rate) not existential proof that prediction is always doomed.

After all, if Black Swans were common enough to make prediction a fool's errand most of the time, the bird of that name would never have led to the book of that name, because everyone would be predicting their failure to predict things.

[sgt101]

I think that a Black Swan is when a new factor appears in your domain. In science we are conditioned from the start to create fair tests in controlled experiments. Control is the fundamental of experiment - and statistics are designed to handle experimental data.

In the real world there are often no controls, and complex systems can be driven by an attractor for a very long time before one morning they are not, and every rule that you have is useless (often worse than useless).

Sources of error are not equal; "Black Swan Error" is unusual in that over time it may be that this source is more important than any other source of data in your domain - the strange attractor that drove the creation of your classifier over the last 20 years may never recapture your function and if that's the case your classifier will be literally the most wrong thing you could have!

[ben_w]

That's certainly one type of Black Swan. Taleb's example of that sort of thing being a Turkey predicting they will be fed (because that is what happened every other day of their life) but who is actually slaughtered.

However, it is not the only type. There is also the stock market, which demonstrates major unpredictability every few years, but which can also be approximated the same way between each of the Black Swans. (And they keep being Black Swans because the gap between them is large enough for people to convince themselves that "This time it's different, this time n̵o̵b̵o̵d̵y̵ ̵w̵i̵l̵l̵ ̵h̵a̵v̵e̵ ̵t̵o̵ ̵b̵e̵ ̵n̵a̵i̵l̵e̵d̵ ̵t̵o̵ ̵a̵n̵y̵t̵h̵i̵n̵g̵ growth will be eternal!")

Edit:

Point is, it generalises as how wrong you are in your predictions, and the closer your estimate of your error rate is to your actual error rate, the better your model is.

[taneq]

I always find the term 'black swan' to be interesting, because where I live, black swans are the rule rather than the exception. I think this just makes the analogy even better, since it highlights how much your ability to predict events depends on your environment.

[danieltillett]

Me too :) It is not a term that get used much here (Australia).

I have to say I rather prefer the black variety over the white.