Hacker News new | ask | show | jobs
by dragonwriter 4357 days ago
If you have decent information on the reliability of your process, there is some argument that, if candidates are close enough in the results under that process that there is a very small probability that the differences are meaningful, a random selection among the candidates might be a better choice than consistently choosing the highest score -- this avoids privileging any systematic bias in the system when the differences are unlikely to be meaningful in what you are using the system as a proxy measure for.

Of course, most places don't actually have much meaningful insight into the reliability of their process so as to enable determining whether and when its outputs are actually meaningful in the first place, they adopt processes that subjectively seem right to the people adopting them, and don't do anything to validate them.