Hacker News new | ask | show | jobs
by tbrownaw 4690 days ago
How do you quantify whether someone is solving a problem the wrong way, just because that way will score better on the metrics?
3 comments

I can remember in the 90s being part of a team where Software Lines of Code (SLOC) was a monthly metric. Highest won a bottle of champagne. People actually stopped writing loops. It got a bit crazy.
I don't have a good answer for you, but let me shoot a question back at you: "How do you determine whether the code you're looking at is good or bad?" Surely, there's an algorithm you run in your head that looks at specific features of the code and analyses them. All I'm saying is that there may be a way to automate that algorithm and use it to drive incentives. Even if the algorithm is as simple as "make sure X and Y unit tests pass."
Assert.IsNull(null);

It's easy to make unit tests pass. To defeat this kind of gaming, you'd need extensive code review of the unit tests. But that circles around to your point: when you look at code there's something that tells you whether it's good or bad. That algorithm can't be based on passing tests, because tests are code too and also need review.

If you had a team of completely selfless developers, you could probably have code-reviews and peer-reviews drive incentives. But as soon as the team suspects it's a zero-sum game and they're competing for a limited prize, they're going to stab each other in the backs, and your best developer will be the prime target. That'll make your best developer leave, and the next-best becomes the target, until all you're left with are lousy developers who are only good at office politics.

I recommend reading http://www.amazon.com/Measuring-Managing-Performance-Organiz... for an analysis of how motivation systems based on measurements often become dysfunctional.

Dysfunction may arise when you are unable to measure all the relevant dimensions of the work being performed. People will often shift their effort to the dimensions that are being measured and ignore the remaining tasks, no matter how important they are. This results in less value being delivered compared to a scenario with no measurement based incentives.

The author mentions software development as an area that is specially prone to dysfunction.

Except, it's impossible to write something that can tell if code ever stops which would seem like a much simpler problem.
Just make the metric to be money and it will work. It takes a big re-org to put the tech tasks in terms of profits, but it's possible to a big extent. Break the software into ownership domains that are peer reviewed and monitored on how much money they contribute. Then sell the software in packages assembled by customers or your internal marketing and solution consultants. This way you encourage the developers to talk to the customers or marketing to collect input and requirements what needs to be done next. Developer can choose to work on something profitable or something cool and obscure, but make no money on it, it's his choice. In both cases, the developer is motivated to be efficient.