[geekjock]

"What should we measure to improve developer productivity?" is a decades-old problem for leaders with no clear solution.

There finally seems to be some level of consensus that output metrics like lines of code, # of PRs, and commits, are an ineffective approach.

Lean metrics like cycle time and lead time can be a helpful high-level diagnostic, but they're far from an indicator of effectiveness or productivity.

A new approach being adopted by many organizations is to focus on the actual experiences of developers... the things that slow them down or frustrate them... and turn these into measurements that guide improvement. I'm the founder of getdx.com where we're publishing research on this: http://paper.getdx.com

[Cardinal7167]

> "What should we measure to improve developer productivity?"

How about nothing? Honest question. I am sick and tired of being treated like cattle. I've never seen a single convincing argument that any metric of my outputs has mapped to some productivity metric in a meaningful way, but I have absolutely seen them weaponized against me and others to punish and fire engineers. All of these metrics have only ever boiled down to a negative reinforcement mechanism, in my experience.

[geekjock]

I'm a developer and am right there with you. But if you're a decades-old corporation with 10,000 engineers, you need some set of signals to help guide improvements to tools and processes, right? This benefits developers, and there should be a set of signals to enable this.

[closeparen]

You need objective, measurable signals if you are concerned that developers might be confused or lying. Otherwise you can just ask us. Most people are happy to tell you what the pain points are in their workflow and how they compare to last year's. Maybe you need metrics to quantify specific complaints like "slow" or "crashes a lot." But I feel like most organizations doing this kind of measurement are looking for some kind of Freakonomics "developers think they want X, but what makes them better is actually Y" when they haven't even bothered to ask about, let alone implement, X.

[impute]

> You need objective, measurable signals if you are concerned that developers might be confused or lying. Otherwise you can just ask us

In an organization with hundreds or thousands of developers, there will be people either lying about how productive they are or genuinely think they are performing above average when they are not. It's like how 80% of people think they're above average drivers.

On the other hand, you may have excellent developers that are overly modest or not loud enough to sell themselves. They are truly exceptional and above the curve and should be rewarded for that. Some of the 20% of drivers that don't think they are above average may actually be above average.

Objective, measurable signals help to find the outliers at the ends of the curve that may otherwise be missed.

[closeparen]

It doesn't matter what percentile developer you are to guide improvements to tools and processes. It matters what slows you down.

[dalyons]

In healthy orgs you are collecting such metrics to help advocate for investing in tooling, automation, infrastructure, process improvements etc. It can give the business case for building/expanding dev tooling and cloud teams. I’ve found it helpful in past jobs… fully weighted dev salaries are so high, at a moderate sized org a little bit of rough efficiency math can make for obvious investment cases.

[Kamq]

> In healthy orgs...

This is, so far as I can tell, ~5% of them.

[pixl97]

So how do you deal with bad developers? How do you even measure if they exist in your organization.

And as much as many of us on HN are developers and like to toot our own horn, some of us suck and don't get better with time. I mean in small organizations this is typically easy to figure out, but in larger organizations it's a problem that can persist for long periods of time.

[Cardinal7167]

> So how do you deal with bad developers?

The same way we deal with good developers? With managers, of course. That's the whole point of their role. Every developer productivity metric I've seen speaks to some disparity between engineering and the C-suite. I don't understand where these products originate that seek to do the manager's job for them but worse. I've been in management roles before, the numbers don't and can't always tell you the full story, no matter the size of the team or the quality of the data.

As always, good people are hard to find.

[impute]

What if the manager is bad? Metrics can help other people identify that a bad manager may not be identifying very good developers.

[ajmurmann]

If the manager is bad, we should address that by discussing metrics to measure manager success and identify three bad managers, work with them on improving and fire them off it doesn't help.

[naasking]

> So how do you deal with bad developers? How do you even measure if they exist in your organization.

Ask the developers that work with them? Who likes working with someone that isn't pulling their own weight, or worse, that is just making your own work harder?

[lylejantzi3rd]

Who cares? No company I've ever heard of has been killed by bad developers. The same can't be said for managers, Vice Presidents or the C-Suite.

[theptip]

The big problem with this line of reasoning is that good developers hate working with bad developers. So if you don't do something about bad developers (which might just mean train them, not necessarily fire them), then the good developers will leave.

[lylejantzi3rd]

In that case, the bad developers are easy to spot. The good developers point right at them. There's no mystery nor are measurements needed.

[Jensson]

That is circle reasoning, if you know which developers are good so you can ask them then you already know which developers are bad.

And asking good developers to identify good developers doesn't work either. Good developers hates working with bad developers, true, but like all other humans they hate working with all sorts of people for different reason so they will point out a lot of good developers as well.

Instead what works is if you have lots of data from diverse group of good developers. That way the personal noise gets cancelled out and you are mostly left with the "I don't like bad developers" signal. This isn't trivial data to get though, especially since people usually choose to work with people like themselves so a team likely isn't diverse enough to get you this kind of data, and people from other teams aren't familiar enough to give good data either.

[pixl97]

You hire 5 developers.

3 of them point at 2 of them and say they are bad.

You fire 2 developers.

Nothing gets done. Turns out the 2 were the good developers.

Congratulations, you succeeded at making another bad metric.

[pixl97]

This is a tautology if I've ever heard one.

If a bad manager keeps only bad developers around it will kill a (software) company just as fast as anything else. It turns out that paying customers like their software to work and be free of data corruption.

[kqr]

I disagree. Under good management, bad developers might progress more slowly than good ones, but they won't outright destroy the product.

[theptip]

Speed == probability of survival for startups. You only have so much runway; better developer productivity means more swings of the bat at finding PMF.

[Jensson]

When you lack good developers stuff like this happens:

https://www.cbc.ca/news/business/sony-cyberpunk-2077-playsta...

Edit with more arguments:

You could of course blame managers for that, but delaying a game since your bad developers aren't capable of delivering isn't really an option either. How many years will it take those developers to get things right? What they would need is to delay the game and replace the developers with good developers, or scale back the complexity of the game to a level those developers can handle.

The company in question choose the latter, they said they will use unreal engine for future games instead of an inhouse engine. So they admitted their developers aren't good enough.

[chenmike]

Paying a non-trivial portion of your engineers high salaries will kill any startup without unlimited money, regardless of what other guardrails you have in place.

[senguidev]

IMO some ~objective feedback of reality is good though. Probably not easy to set up without having it quickly turn into a negative reinforcement loop indeed. But let's not forget that, with the right supporting culture, it can be helpful (pleasant?) to the dev to have some metrics. I admit "how to do that?" remains unclear.

[yamtaddle]

But if you don't measure them—or if, god forbid, you give them their own offices with doors—they might start to think they're in the professional/upper-middle class, not the middle class!

[hcarvalhoalves]

This doesn’t work, because then your company would have to measure your value by the value you deliver. That would probably mean having to compensate people like you way better, because software engineers can create immense value. Of course this can’t happen, as we apparently live in a universe where only management have bonus based compensation.

[astrange]

Who’s we? FAANG engineers have bonus and stock compensation.

[rqtwteye]

“the things that slow them down or frustrate them”

Whenever management makes a new attempt at getting more precise estimates I always tell them the only way to get things done is by doing them. So the number one goal should be to remove obstacles, simplify processes and make sure tools work to our benefit. For some reason nothing much ever changes and the question for better estimates returns. It’s pretty weird.

[quickthrower2]

Anything that is routinely easy to estimate can probably be automated away. And so will your competition.