Hacker News new | ask | show | jobs
by athenot 2103 days ago
Poor metrics are meaningless regardless of what you try to measure.

In the example of documentation a better metric might be a composite value of

(i) how many wiki articles a developer has written, weighed 0.33 and

(ii) how useful on a numerical scale his/her peers rate the documentation produced, weighed 0.67.

This is just an example but it's a composite of perceptual and objective, with the perceptual being a lot more important (and to prevent gaming the system by writting lots of useless articles).

2 comments

The way I've experienced it is that when a measurement becomes a metric, it ceases to be a good measurement. When people's bonus is tied to a measurement, they will find creative ways to influence that measurement which, in many cases, defeats the purpose of what it was trying to measure in the first place. It's not simply a matter of choosing the right measurements. It's also a matter of how you incentivize those measurements.

When I worked at Microsoft one of the teams I was adjacent to had a metric on number of new apps in the Windows Phone app store. So the teams went out and got a bunch of college students to build shitty apps in bootcamp style working groups. Suddenly the number of apps isn't good enough, so they added review metrics. Now those teams add a "let's all rate each other's apps" portion to the bootcamp taking you even further away from the results you're trying to obtain.

It's all in the details. If I get to pick who rates "how useful", this translates into a general "how well liked by my colleagues am I" (general problem with 360* feedback; I'm not saying it's useless, but it does have its limitations)