[xmddmx]

There's a meta-level of irony here that's important to note.

TFA is defending the use of AI, and it very clearly (to me) used AI to analyze the data and present the results.

In doing so, the author used statistics in a way they do not appear to understand, and ended up making numerous false claims (you can see the thread discussing these here https://news.ycombinator.com/item?id=48417626 )

In short, the study doesn't have sufficient statistical power, and is making "no difference" claims that aren't justified.

The meta-irony is this: the author used an LLM to interpret data in this study, and seems to have made the same category of mistake (confidently asserting falsehoods) that the study was supposed to be investigating (confidently submitting bad commits to the rsync project).

[simianwords]

The meta-meta level irony is that the reaction to this post is based on vibes and misunderstands the point of the article to wage ideological warfare -- much the same way the original github issue was written.

[classified]

AI is so much like a religion. There is nothing you can say to a believer that will make them question their believes. Or more generally, you cannot reason anyone out of something that they want to believe.

[newsoftheday]

AI is nothing like religion. People behave similarly to AI when debating their favorite sports team, or for Java coders, Checked vs Runtime exceptions.

Religion is about faith and what people feel and sense as much as believe.

[Joel_Mckay]

It gets pretty dark if you pull that thread of reason. =3

https://en.wikipedia.org/wiki/The_True_Believer

[logicprog]

The statistical methodology I used is mine. As is the interpretation. Completely. To the degree that I misunderstood statistics (and it is under debate even in the thread you link, and the people accusing me of misunderstanding statistics there are universally misrepresenting my point, which is to point out a total absence of evidence for any difference, not to prove the null hypothesis) that's on me

[MichaelDickens]

FWIW I understood your point just fine. It seemed to me that you made a clear enough distinction between "evidence that Claude didn't increase bugs" and "no good evidence either way".

[runarberg]

I don’t think that point is clear at all, at least in this comment sections you can find a few posts like these:

> Did Claude increase bugs in rsync?

>

> TFA answered this, the answer is “no”.

https://news.ycombinator.com/item?id=48423811

[karagenit]

It seems like based on e.g. [1] the article originally made some stronger claims about “no difference in bugs” that have been corrected. I agree that now it seems fine, but those edits might be why it feels like some commenters read a different article than you.

[1]: https://news.ycombinator.com/item?id=48418186