[edanm]

1. We're very bad at measuring developer productivity. We've been trying to do it for a long time, and really have very little to show from it from my POV.

2. That said, almost all the people who "want to see a study" don't make sense to me. I don't remember anyone insisting on seeing a study that shows that writing Python is more productive than C; people just used it and largely agreed that it was. How many studies show that git (or other DVCS) are better than the things that preceded it? I don't know if any exist. I do know that nobody was looking for studies before switching to git.

I don't ever remember seeing any new technology in software development for which people demanded studies before adopting it. They just assumed that if the professional developers they trusted to build their software said something was better, then it was — a correct assumption IMO.

Now, we're seeing a technology which most professional developers — that have used it seriously, at least — insist is orders of magnitude better than anything else that's come before it. And suddenly developers can't be trusted? Suddenly, when the claimed effect is orders of magnitude bigger than almost any other new technology, developers are biased and incapable of making this kind of determination?

I really don't think that's a serious position to hold.

[000ooo000]

>Now, we're seeing a technology which most professional developers — that have used it seriously, at least — insist is orders of magnitude better than anything else that's come before it.

You can't just assert this. I could equally-baselessly say most professional developers have used LLMs and find them, overall, more trouble than they're worth. Except it's not totally baseless because I think that was actually a result of a study, IIRC.

[000ooo000]

GP's bio:

>I'm [...] at $x, a frontier AI Security company

I really should check these before I bother engaging with posts boosting AI

[khuedoan]

But we didn't have pressure to switch from C to Python & solved it down our throats by management, or social media telling us if you don't use Python you're getting left behind, did we?

In C vs. Python case, we know about technical trade-offs and when to use what, but in AI productivity neratives, we keep pretending that technical or cognitive debt created by AI doesn't exist.

Sure, person A can be 20% "faster" and suggest that this tool increases productivity by a magnitude, but if it costs person B 50% more time to review A's slop or clean up A's mess, the team's productivity doesn't really increase.