[r0uv3n]

The discussion between Nicolas Boneel and the author in the comments of the article is interesting and Nicolas expresses the doubts I had when reading this. The whole point of the DK effect is that people are bad at estimating their skill, so if you assume that they randomly guess their skill level then of course you will replicate the results.

The correct model for a world without DK should be something like (estimated test scores)=(actual test scores)+noise, and then the only form of spurious DK you'd expect is caused by the fact that there's a minimum and maximum test score. But this effect would be proportional to the variance of the noise, and I assume the variance on the additional dataset is too low to fully understand the effect seen there.

Also, in this model on average everyone should still guess correctly in which half of the distribution they are, but even the bottom quartile seemed to estimate their abilities as above the 50th percentile

[jampekka]

The correct model is probably (estimated test score + estimation noise) = (actual test score + test noise). The test contains a random element, e.g. guessing, that the person can't estimate.

https://en.m.wikipedia.org/wiki/Regression_dilution

https://en.m.wikipedia.org/wiki/Errors-in-variables_models

[svnt]

Just because the data appear random doesn’t mean you’ve gotten at the cause though.

From those charts it could equally be low skill throughout, or something nuanced like lack of skill at estimating at the bottom, improving skill in estimating through the middle, and high skill and learned modesty at the top.

[Jensson]

> Also, in this model on average everyone should still guess correctly in which half of the distribution they are, but even the bottom quartile seemed to estimate their abilities as above the 50th percentile

Depends on the noise applied. If the noise is -10% to +100% for everyone then you get roughly the graph Dunning-Kruger got. So there is no reason to believe that the best are better at estimating their abilities, just that you can't estimate your own rank as better than the best.

[tempestn]

That's a great observation. For what it's worth though, it does seem logical to me that the best would also be best at estimating their skill. Not necessarily because they're better at it per se (though there's likely some of that too, for the reasons originally posited by D-K), but also because they have an easier problem to solve. When you know something well, it's fairly obvious that that's the case. (Think of the experience of acing a math test. It's entirely possible you'd know you answered everything correctly.) When you struggle somewhat though, it's much more difficult to estimate how much you're struggling compared to how others would fare.