[sgillen]

This is very interesting because I see a lot of AI detractors point to the original study as proof that AI is overhyped and nothing to worry about. In this new study the findings are essentially reversed (20% slowdown to 20% speedup).

[selridge]

I think their old findings were hard to treat as gospel just due to the kind of comparison + the sample, but this new result is probably much noisier.

It’s hard to make reliable, directional assumptions about the kind of self-selection and refusal they saw, even without worrying about the reward dropping 66%.

[fxwin]

fwiw i think the interesting part about the original study wasn't so much the slowdowm part, but the discrepancy between perceived and measured speedup/slowdown (which is the part i used to bring up frequently when talking to other devs)

[simonw]

AI detractors loved that previous study so much. It seems to have been brought up in the majority of conversations about AI productivity over the past six months.

(Notable to me was how few other studies they cited, which I think is because studies showing AI productivity loss are quite uncommon.)

[smohare]

Or maybe there’s just not that many good studies, period?

A lot of them barely rise above the level of collected anecdote, nor explore long term or more elusive factors (such as cross-system entropy). They’re also targeting an area that is fairly difficult to measure and control for.

[ej88]

not enough people look at the slope, just the coords