[bakugo]

> like the double standard between anti-AI and pro-AI claims, one of which gets to make claims based on cherry-picked anecdotes, and the other which must produce rigorous studies

This statement is honestly so ridiculous that I felt it didn't warrant a direct response, but here's one anyway: AI enthusiasts have been proudly proclaiming for literal years that AI makes them 10x as productive based on cherry-picked anecdotes with zero empirical evidence to back it up. It's way, way too late to claim hypocrisy here. As I stated under the original submission about this topic, irrational anti-AI behavior is usually just an equal and opposite reaction to irrational pro-AI behavior.

> I rewrote all the prose in the entire document just to satisfy critics like you.

And that doesn't help. If anything, editing the AI output to make it read less like blatant slop just comes off as deceptive, like you're trying to hide the fact that the analysis was AI generated. Looking at the commits, you were adding more AI generated text less than 2 hours ago[0] before quickly editing out one of the most blatantly sloppy sentences I've ever read[1].

Regardless, the final contents of the article are not the main issue. Even if we ignore the bias clearly on display there, the premise alone is enough to dismiss the entire thing as heavily biased and chasing a pre-determined conclusion - of course someone who is so dependent and trustful of AI that they decide such an analysis on the bugginess of AI code should itself be written by AI is going to steer the conclusion towards "actually AI code is good and you luddites are overreacting". The entire concept is so tone-deaf that failing to notice it or predict the criticism before publishing is enough to prove the bias.

[0] https://github.com/alexispurslane/rsync-analysis/commit/e029...

[1] https://github.com/alexispurslane/rsync-analysis/commit/740b...

[logicprog]

Also.

> This statement is honestly so ridiculous that I felt it didn't warrant a direct response, but here's one anyway: AI enthusiasts have been proudly proclaiming for literal years that AI makes them 10x as productive based on cherry-picked anecdotes with zero empirical evidence to back it up.

Let's go back to remedial classes on this one.

"I have found that [tool] has made me more effective" is what we call lived experience. It is an "i" statement communicating something about the person’s life. It does not require evidence by default, and you are a crazy person if you call bullshit without good reason, because many "I" statements are epistemically justified in ways that can't be empirically demonstrated or require tacit knowledge.

"[tool] has been buggier since [change]" is a falsifiable claim; you need to actually provide evidence for believing it, and what I'm showing is literally that there isn't any.

[logicprog]

> This statement is honestly so ridiculous that I felt it didn't warrant a direct response, but here's one anyway: AI enthusiasts have been proudly proclaiming for literal years that AI makes them 10x as productive based on cherry-picked anecdotes with zero empirical evidence to back it up. It's way, way too late to claim hypocrisy here. As I stated under the original submission about this topic, irrational anti-AI behavior is usually just an equal and opposite reaction to irrational pro-AI behavior.

I'm talking about the double standard on the anti-AI side about what evidence should count, not some vague industry-wide epistemic standard, whatever that means. I'm aware LinkedIn Lunatics and Steve Yegge are also being crazy. And it seems to me that even your response here is engaging in a bit of a double standard, or something akin to it, in that you think the irrational anti-AI behavior should be given a pass — and the conclusions perhaps even taken seriously — just because pro-AI people did it too.

> And that doesn't help. If anything, editing the AI output to make it read less like blatant slop just comes off as deceptive, like you're trying to hide the fact that the analysis was AI generated.

Okay, so, if I don't spend the time to write everything myself, that's bad because it's AI slop. If I do rewrite everything myself, then it's evidence of deceptiveness... despite being asked by multiple people to do that, and being extremely explicit about my methods and process and the commit history being (as you've shown), very public.

Also, the AI-generatedness of the text doesn't mean the analysis is AI generated, in terms of what was actually done. That's a category error.

> Looking at the commits, you were adding more AI generated text less than 2 hours ago[0] before quickly editing out one of the most blatantly sloppy sentences I've ever read[1].

The second commit literally says that that was my prose it was fucking with by adding slop. It's just that me adding my prose, and it adding slop to it, were in the same previous commit. Additionally, my process is often giving it exactly what I want to say, more or less, and having it HTML-format it and insert the templated numbers and UI widgets around that text.

But again, even if I'm spending the time to read through and edit everything it's writing to de-slop it, then I'm clearly also reading it through enough to make sure the analysis makes sense, and is accurate; how is that not enough "effort" for you, if effort is supposed to be a proxy for verification?

> Even if we ignore the bias clearly on display there, the premise alone is enough to dismiss the entire thing as heavily biased and chasing a pre-determined conclusion - of course someone who is so dependent and trustful of AI that they decide such an analysis on the bugginess of AI code should itself be written by AI is going to steer the conclusion towards "actually AI code is good and you luddites are overreacting".

That's not ignoring the bias, that's literally restating that you think the bias is there. But if you really think that my bias meaningfully "steered the results," then show me how that happened. Tell me how you would've proven the Claude releases were meaningfully worse, or unusual, at all, or how the methods I chose biased the data against that result, or literally anything except shifting the goalposts and using accusations of "bias" as a get-out-of-jail-free-card.

> The entire concept is so tone-deaf that failing to notice it or predict the criticism before publishing is enough to prove the bias.

And you're so committed to your preconceived notions that anything made with AI must be bad, wrong, or not worth your time, that you'll spend your entire time begging the question ("it's made with AI, therefore it's wrong") and shifting the goalposts instead of engaging meaningfully.

Also, I certainly predicted the criticism (in general, anyway, to the fact that it was made with AI; not the prose being AI) but I made it this way anyway, because if someone is so AI-blinded that they can't read and evaluate the actual metrics, methodology, and provide meaningful criticism to it, and instead can only see that it was made with AI, and they're so it doesn't matter.

Nothing you have said makes the analysis wrong. At this point, you're essentially just resorting to ad homenem and begging the question.

[bakugo]

> If I do rewrite everything myself, then it's evidence of deceptiveness... despite being asked by multiple people to do that

I don't know who asked you to do it. I wouldn't have done it. Personally, the original intent matters far more to me. You intended to submit an AI-generated article, defending AI, to be read by humans. Anything short of taking the article down and rewriting the entire thing from scratch doesn't meaningfully change that.

> Additionally, my process is often giving it exactly what I want to say, more or less, and having it HTML-format it and insert the templated numbers and UI widgets around that text.

Sorry but you're just further proving my point here. You are so deeply invested in AI that even just manually writing some English text into a static HTML file is something you consider to be below you.

Imagine going back in time 5 years and telling someone: "In the future, nobody uses text editors. On the rare occasion that we actually want to write something to a text file verbatim, we instead recite the text to a complex artificial intelligence algorithm that uses large amounts of computing power to process said text and then recite back a command that writes the text to a file. Sometimes the algorithm decides to be a smartass and change our words or add an extra quip, but that's all part of the fun."

> That's not ignoring the bias, that's literally restating that you think the bias is there.

I was referring to the bias within the actual text of the article vs the inherent bias displayed by the very concept of an AI-generated article defending AI. Passages like these:

> The thread did not stop at words. As is typical for anti-AI users, it eventually escalated to fantasies of violence

Make it fairly obvious that you went into this project with the primary goal of proving such people wrong, possibly backed by a sense of moral superiority relative to a few weirdos on the internet who took things too far (such individuals are present in every online discussion that gets big enough, and their actions do not represent the whole).

> And you're so committed to your preconceived notions that anything made with AI must be bad, wrong, or not worth your time

"Bad" or "wrong" may be subjective, but it's definitely not worth my time, no. If you didn't consider it worth your time to write it, why do you believe it's worth someone else's time to read it? Again, it doesn't matter if you went back to rewrite parts of it after being criticized, as that doesn't change the original intent.

Submitting an AI generated article and expecting meaningful human responses only makes sense if you consider your own time to be worth more than that of others. Do you?

[shawndumas]

i feel like OP put their money where their mouth was. they dug in and did the analysis. they also capitulated and rewrote the lest interesting and easiest part of the post; the prose.

i also am seeing them engage aptly with constructive criticism and adapting the material while handily dispatching the non-constructive critiques. most of which amounts to a colossal missing-of-the-point.

they have made no out of proportion claims, no non-recreate’able analysis, used exactly the correct tools, and, frankly have addressed all of your points

i am not sure you’ll agree with anything i’ve said either so feel free to misunderstand me too

[logicprog]

Okay, so let me get this straight. Because I used AI to, among other things, write the prose of the original draft of this article, all of the days of effort researching and carefully thinking through the metrics I would use, and the methodologies to analyze them, and rewriting the entire analysis multiple times from scratch based on specifically asking people in my life who are qualified in statistics what I should do and trying to come up with the fairest analysis I could with the little data available doesn't matter at all? The post just have some sort of essential AI nature that makes it low effort just because one aspect of it didn't have sufficient effort put into it for you personally?

And once it's originally posted, it doesn't matter the great extent I go to address metrics and methodological critiques in order to ensure that the data is as robust and helpful as possible. And the effort in writing and refining my prose and the organization of the report in response to people's complaints and criticisms because I do value their time. And when people told me the AI prose was bad, I spent two hours to to make sure that it was something people would want to read, that doesn't matter at all? It's only the original intention that matters. So you just have this arbitrary cutoff point for what counts towards my intentions in the post and my character. No allowance for learning or adaptation, and the fact that I'm clearly committed to putting a lot of effort into making this something that is useful and pleasant to read for people, I just didn't do it for the first draft originally, doesn't matter, only the original version matters?

And more than that, you're not going to actually deal with the substance of the issue, the actual calculations and methodology and conclusions that I came to, instead, the only semi-substantive critique you're going to make of the post is to tone police me and dance around the real issues, as if you're afraid of ever touching them?

The best argument you could make that my bias actually influenced my conclusions would be to point into the methodology and metrics where I did that. I made it all extremely open and transparent and auditable both by describing it in extreme detail in the post and by providing all of my source code and the ability to build the database it runs on from scratch. If there was an actual flaw or bias that my intentions going into this created your biggest possible Smackdown, your best weapon in your arsenal would be to actually point that out. But instead, again, you're just tone policing me. but a polemical style in the presentation of an objective statistical analysis does not in the least undercut its accuracy. Have you considered that my polemic became so fiery, in fact, precisely because I ran the tests and found how non-existent the evidence was for this outrage and that's what made me angry? No, you didn't because you saw some words that hurt your feelings and now you won't listen to facts.

[bakugo]

If you really did spend days on research and methodology (which, to be clear, I'm not denying), that just makes it all the more disappointing that you decided to cap it all off with a long AI generated article. The article is what I'm focusing on because it's what you expect other people to actually read, and it's what you submitted here.

Ultimately, I'm just trying to get you to understand how this decision undermines the presumed goal of trying to convince the anti-AI crowd that they're wrong. It's simply not fair to expect humans to engage with the article in good faith when the article itself was not written by a human in good faith, regardless of its contents or the numbers it's based on. If you still disagree, so be it, I have nothing else to argue.

And for the record, I didn't engage with the methodology itself or its merits because I don't believe this question can be answered via an automated statistical approach, or really any sort of objective approach. The only way to truly evaluate the quality of AI generated code is for a skilled developer who is at least moderately familiar with the codebase to carefully analyze each commit, understanding what it does and looking for dumb mistakes that a human likely wouldn't have made in the same situation. But it's very unlikely that anyone will waste their time on that, and the conclusion would still be subjective anyway.