| I worked with Ferrucci in BW. The internals of the employee rating system that preceded him were hilarious. In short: Everyone is encouraged to rate anyone else in a variety of categories, as often as possible. Every rating is public. You know who is rating you, and how they did it. Those ratings are put together to get a score in every category, and can be seen by anyone. It's your Baseball Card. The problem is that not everyone is equally 'credible' in their rating. If I am bad at underwater basketweaving, my opinions on the matter are useless. But if I become good, suddenly my opinion is very important. You can imagine how, as one accumulates ratings, the system becomes unstable: My high credibility makes someone else have bad credibility, which changes the rankings again. How many iterations do we run before we consider the results stable? Maybe there's two groups of people that massively disagree in a topic. One will be high credibility, and the other bad, and that determines final scores. Maybe the opinion of one other random employee just changes everyone else's scores massively. So the first thing is that the way we know an iteration is good involves whether certain key people are rated highly or not, because anything that, say, said that Ray is bad at critical thinking is obviously faulty. So ultimately winners and losers on anything contentious are determined by fiat. So then we have someone who is highly rated, and is 100% aware of who is rating them badly. Do you really think it's safe to do that? I don't. Therefore, if you don't have very significant clout, your rating of people should be simple: Go look at that person's baseball card, and rate something very similar in that category. Anything else is asking for trouble. You are supposed to be honest... but honestly, its better to just agree with those that are credible. So what you get is a system where, if you are smart, you mostly agree with the bosses... but not too much, as to show that you are an independent thinker. But you better disagree in places that don't matter too much. If there's anything surprising, is that more people involved into the initiative stayed on board that long, because it's clear that the stated goals and the actual goals are completely detached from each other. It's unsurprising that it's not a great place to work, although the pay is very good. |
Tangential to your point but this is actually a solved problem and you only have to run once. You might even recognize the solution [1] [2].
[1]: https://en.wikipedia.org/wiki/PageRank [2]: https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors