[frostlynx]

24% of increased productivity (yes, this is assuming of course that the “proxy” of merged PRs reflects productivity) is actually a pretty big deal. Given the salary of developers, this translates to tens of thousands of dollars per year, per developer.

My guess is they used # of PRs as a measure as it’s easy to obtain, while other measures are hard, may be due to other factors, etc.

FWIW I saw a similar number for myself, around 30% more PRs in the last 6 months, compared to the 6 months before that (I picked up agentic coding around at the start of the year). And a similar increase for closed issues.

In my case this clearly doesn’t translate to as much value for the organization, or rather, it’s hard to say, as many of those PRs were things I wouldn’t even have done without AI support. This means they were low priority. However, many were of the cleanup/refactor type, so they might result in speedups later.

[Arainach]

24% PRs isn't 24% more productivity. Lines of code isn't productivity, and neither is CLs landed. What's the feature velocity of the team? How much time is being spent on rollbacks, outage responses, etc.?

Here's a quick hack to triple your PRs landed: Land a PR, then land ANOTHER PR undoing that one when you realize it was full of bugs, then land the PR again once you realize management doesn't care about quality, they just care about the number of PRs landed.

[meling]

The 24% more PRs was merged, not «landed». Presumably after human code review (but I didn’t check the paper…) Not sure about this particular study, but I «think» my own productivity (coding wise) has improved by more than 30%. In many ways I wouldn’t even have started on many of the things I’ve completed these last six months because I would have viewed their effort to be insurmountable within my time budget as a professor (with all my other duties).

[onion2k]

I've also seen an increase in merged PRs, but it coincided with developers opening smaller PRs. In other words, AI made devs break work down more so they opened more PRs for the same work. That's still good, because smaller increments are better, but there' no actual increase in coding productivity, and it means the context-switching burden from review work went up e.g developers slowed down in a different area.

[regularfry]

I'm yet to be convinced that number of PRs as a productivity metric is any less flawed than counting lines of code.

I can believe - easily - that there's a real uplift here, but attaching any meaning to the 24% number at all is a massive overstep.

[gojkoa]

PRs measure effort, not outcome, so it’s a measure that their developers+ai spent 24% more effort than just devs previously (assuming the average work that went into a PR didn’t change, which is also a big assumption. maybe those PRs were smaller or more complex on average than before). so the uplift isn’t really tracked here. time to market, average cycle time per change, number of incidents per ticket released etc would be more meaningful metrics for uplift

[pushplay]

> In my case this clearly doesn’t translate to as much value for the organization, or rather, it’s hard to say, as many of those PRs were things I wouldn’t even have done without AI support.

That's not necessarily bad. It could be a sign of effective prioritization. If you're good at working on the most important thing, and suddenly find yourself 24% more productive, what extra are you working on? The things that wouldn't have quite made the cut before.

[btown]

Given that Microsoft's overhead to actually landing features has, from what I've heard, long been more about dodging the cross-org and inter-org "guns" famously depicted in [0] and [1] than coding time... a 24% increase in merged PRs is massive!

[0] https://static.ma.nu/publications/images/2013.07.12_new_york...

[1] https://newsletter.pragmaticengineer.com/i/138252015/drawing...