[januzis]

It seems to me there are two sides to engineer's performance: the ability and the productivity. Ability measures how complex tasks can an engineer solve and how well can he/she execute, and the productivity measures the actual amount of work done. Able programmer is not necessarily productive, and productive programmer might not be able to do tasks of high complexity.

As a technical lead, I feel that I'm able to judge the ability of individual team members, but I'm having a hard time objectively judging productivity. Simple count of PRs doesn't really tell the whole story, and some tasks look simple in hindsight, when in reality it took a lot of effort to find a good solution. There are also a lot of other complications I'm not going to dive into, but the end result is that it's hard to have an objective productivity evaluation based on the engineer's output only.

I'd be interested to know how other people evaluate individual productivity?

[jrumbut]

People say that no metric works but I think almost any metric works, for example PR count.

You, a human being, would never actually confuse the engineers with 0 PRs because they spent the last month playing online poker with the engineers with 0 PRs because you entrusted them with developing an automated deployment system for a legacy application and starting a mentorship program.

Many metrics will spot the outliers. But what is the business value of knowing when an engineer rated 82.13 vs 84.51?

[sam0x17]

Yeah but what about the highly skilled employee who spends 90% of their time playing online poker and spends 10% of their time producing output on-par with other people in the org that work 100% of the time. There is almost always one of these in any given org.

[januzis]

I would actually be OK with it, as long as I can objectively say that even though someone is slacking most of the time, he/she is very productive in short bursts, so the overall productivity is on par with the team average.

The problem is that if you don't have an objective approach to evaluating productivity, all sorts of biases come in, e.g. if I see someone coming to the office at 11am and leaving at 4pm, I would subjectively rate their productivity lower, even though objectively the results might be the same as for someone who comes in early, and leaves late.

[Smithalicious]

Well are they getting paid 10 times as much as the others?

[januzis]

Yes, the extreme cases are easy to spot, but the nominal cases are not that minuscule, more like one developer being twice more productive than the other, which, I feel, should be recognized and rewarded accordingly.

[JamesBarney]

This is especially tough because one important development skill is the ability to solve complex problems with simple solutions. This can sometimes make them look less productive because they've made a hard problem look easy.

One technique I've found that helps a little is having everyone individually estimate tasks before hand. Over time you can notice who is completing tasks faster or slower than the average. Of course this is just one data point that needs to be weighed against a variety of others.

[januzis]

Measuring against estimates incentivizes over-estimating, especially if the team is small, and team members know roughly who will do what during the estimation process. Big complicated tasks also tend to be under-estimated, so they would be avoided, because they would usually influence the perceived productivity negatively.

I would like to find a feasible approach to productivity evaluation using the output only (PRs, reviews, basically all the data points the article mentions), because I feel that’s the only way of creating an environment, where team members can proactively take a bit more time when it benefits the end result, without fearing any negative repercussions.

[JamesBarney]

I don't think developers spend that much time gaming metrics unless they are objectively measured against them.

And you would notice if someone's estimates are always higher for the work they are assigned compared to others.

[yowlingcat]

You talk to them and the people they work with or whose work depends on them. How quickly, autonomously and completely do problems get solved?

Metrics are a distraction unless in the service of answering that question.

[januzis]

That’s the default, I guess, my main concern is that it’s very subjective.

[yowlingcat]

Sure -- some parts are, but some parts aren't. It's intrinsic to the nature of working in a team or coaching a team. If you notice weak spots consistently, is it a fluke? Probably not. If you notice complaints about the ability of the team, division and org to collaborately smoothly, is that a fluke? Again, probably not. By the time you fine-tune all the weak spots properly, you have a very high functioning team. But it takes a lot of iteration.