[thebigspacefuck]

The METR study showed even though people feel more productive they weren’t https://arxiv.org/abs/2507.09089

[knes]

the MTR study is a joke. it surveyed only 16 devs. in the era of Sonnet 3.5

Can we stop citing this study

I'm not saying the DORA study is more accurate, but at least it surveyed 5000 developers, globally and more recently (between June 13 and July 21, 2025) which means using the most recent SOTA models

[rsynnott]

> I'm not saying the DORA study is more accurate, but at least it surveyed 5000 developers, globally and more recently

It's asking a completely different question; it is a survey of peoples' _perceptions of their own productivity_. That's basically useless; people are notoriously bad at self-evaluating things like that.

[capnrefsmmat]

It didn't "survey" devs. It paid them to complete real tasks while they were randomly assigned to use AI or not, and measured the actual time taken to complete the tasks vs. just the perception. It is much higher quality evidence than a convenience sample of developers who just report their perceptions.

[bopbopbop7]

Yea cite the study funded by a company that invested billions into AI instead, that will surely be non biased and accurate.