[verzali]

That post literally says the results are unreliable...

[lars512]

...and in particular it says that one of the reasons is that developers are refusing to participate in the non-AI branch, and when they do, changing what tasks they select to those where AI would be less useful.

Overall this suggests to them that the current speedup is likely greater than what the study could measure.

[dofm]

It might suggest that but they can’t back it up because the study is broken and perhaps forever unrepeatable.

Like, what people are saying is, “That old study was wrong! They did a new broken study that overturned it!”