[groby_b]

Here's the thing every discussion around this tries to weasel around: All else being equal, yes, more PRs is a signal of productivity.

It's not the only metric. But I'm more and more convinced that the people protesting any discussion of it are the ones who... don't ship a lot.

Of course it matters in what code base. What size PR. How many bugs. Maintenance burden. Complexity. All of that doesn't go away. But that doesn't disqualify the metric, it just points out it's not a one-dimensional problem.

And for a solo project, it's fairly easy to hold most of these variables relatively constant. Which means "volume went up" is a pretty meaningful signal in that context.

[SpicyLemonZest]

The problem is that these caveats, while tolerable in some contexts, make the metric impossible to interpret for something like Claude Code which is (I agree!) a huge change in how most software is developed.

If you mostly get around on your feet, distance traveled in a day is a reasonable metric for how much exercise you got. It's true that it also matters how you walk and where you walk, but it would be pretty tedious to tell someone that a "3 mile run" is meaningless and they must track cardiovascular health directly. It's fine, it works OK for most purposes, not every metric has to be perfect.

But once you buy a car, the metric completely decouples, and no longer points towards your original fitness goals even a tiny bit. It's not that cars are useless, or that driving has a magic slowdown factor that just so happens to compensate for your increased distance travelled. The distance just doesn't have anything to do with the exercise except by a contingent link that's been broken.

[groby_b]

> But once you buy a car, the metric completely decouples, and no longer points towards your original fitness goals even a tiny bit.

True, but if what you care about is "how quickly and safely can I reach a given goal", distance traveled over time is a great initial indicator, and accident rate will help illuminate.

The question "does AI help me move faster towards a goal, at the same quality standard", is relatively easy to judge in a solo project. As long as you verify equivalent standards, and don't play in an area you don't know at least - folks have a pretty clear understanding of their own productivity if it's a familiar thing.

[anukin]

Can you define what “all else” means here?

PRs or closed jira tickets can be a metric of productivity only if they add or improve the existing feature set of the product.

If a PR introduces a feature with 10 bugs in other features and I have my agent swarm fix those in 10-20 PRs in a week, my productivity and delivery have both taken a hit. If any of these features went to prod, I have lost revenue as well.

Shipping is not same as shipping correctly with minimal introduction of bugs.

[groby_b]

"All else equal" means that PR volume is a signal that needs to be read in context a number of other metrics, as well as qualitative feedback.

You're absolutely right that PRs fixing things that a previous PR broke is a negative. Same for PRs implementing work not needed, or driving up tech debt.

"You're productive because you have lots of PRs" is a mistake without that context. But so is "You produce very little PRs, but that's fine, we shouldn't look at volume".

It's not a performance metric. It is an indicator worth following up. And there's a lot of reflexive "bad metric" arguments blanket dismissing that indicator.

Does that help explain?

[mememememememo]

Number of integration tests might be a good metric (until you announce that it is the metric then like every other metric, inc. profit, it becomes useless!)

For profit failing as a metric, see: Enron.

[sarchertech]

> All else being equal, yes, more PRs is a signal of productivity.

Yeah but all else isn’t equal, so unless you’re measuring a whole lot more than PRs it’s completely meaningless.

Even on a solo project, something as simple as I’m working with a new technology that I’m excited about is enough to drastically ramp up number of PRs.