[kneebonian]

Here's my concern with this. Is that the AI is good at generating "mostly correct" code, that will have subtle errors, so less skilled coders just take the output of whatever the LLM spits and assume it's correct, later it turns out there is a bug. The person who submitted the code doesn't understand the code, because they didn't bother learning it just that it was mostly correct. Instead of discovering and fixing the source of the error they just go in and discover where the error is output and slap some more AI generated code over it to squash the bug.

Now you may be saying "what's the big deal with that? That happens all the time all over the place."

The issue is that the biggest thing AI brings is it allows you to generate lots of code, very fast. So we have more and more code, that is less and less understood by any human being.

Eventually no one has any idea what the code is doing in the first place, and what ends up happening is the Cult Mechanicus of Warhammer 40k we become tech priests running around performing arbitrary and inane rituals to appease "the machine spirit" because no one anywhere has any idea how the dang program actually works.

Like we've all heard horror stories of the Oracle DB code base, imagine if that started to become common. I mean people talk about how AI helps them generate code faster, but to be honest I find most issues are the result of too much code, rather than too little.

My point is I grow concerned about the ability of AI to generate code and the effect that is going to have on increasing the amount of code while reducing understanding.

[jmnicolas]

Here is an anecdote to illustrate how "The Machine TM" tried to get me fired.

For context, Microsoft introduced a "AI revamped Intellisense" in Visual Studio (not Code) a few months ago. It's not Copilot, it just makes suggestions but only for your next line of code. It's Okish most of the suggestions are good, it saves time on repetitive tasks.

Yesterday I was writing some code to disable some customers. I had a list of all customers and a list of customers to disable. Intellisense offered to pass the list of all customers to the method that will disable them. If I wasn't paying attention, next Monday (and the rest of the week really) would have been quite "entertaining".

[nixpulvis]

> Is that the AI is good at generating "mostly correct" code, that will have subtle errors, so less skilled coders just take the output of whatever the LLM spits and assume it's correct, later it turns out there is a bug.

Not unlike the code generated by the fresh out of bootcamp, careless rookie, who isn't trying to make things work so-much-as collect a paycheck. But hell, at this point I need a job too, maybe I should swallow my morals and submit AI solutions as well?

[ModernMech]

Sure, but I think the imagined hellscape is one where the entire stack of developers don't know what they're doing.

It's like the ostensible toaster company that has outsourced design, engineering, manufacturing, marketing, sales, HR, everything I’ll the way to the janitor. The question then is: what does the toaster company even know about making toasters? If you play this game long enough, then you arrive at a point where entire sectors operate this way. Keep going and the entire economy works this way. At some point we have to stop outsourcing, or we'll lose a lot of knowledge when the original creators of that knowledge kick the bucket. At this point we can't rely on that knowledge being encoded in language models.

[Forgeties79]

My hope is this will mostly translate into being able to distinguish between who really knows what they're doing (the AI is a tool to be used for some tasks and its work is to be scrutinized like anything I'd find on GitHub or elsewhere) and those who are barely hacking it in the field, but I also worry about what you're describing.

Like color/exposure auto-balancing tools in video editing. They can be really useful and can often produce solid color/exposure results! But most of the time they only get you 80% of the way or sometimes just spit out really ugly results. Ultimately auto-balancing didn't ruin editing - mediocre editors can't get by on it - so hopefully we don't see that here!

[leobg]

So how about just prompting it with the task of not only producing the code, but also test cases to go along with it?

Somebody please make a SaaS out of this idea, with a fancy tailwind sales page and a “generous free tier”. :-P

[bathtub365]

Now you have to check generated code and generated test cases for correctness, instead of just code.

[tuxracer]

yes that's true, and sincerely keep going with that maybe another pass could do code review

maybe an elaborate workflow, but each individual step in that workflow isn't insurmountable

if you cannot get the correct answer in one shot, instead a workflow is required, we should just write off this whole thing as useless or setup that workflow? will let the reader decide

[908B64B197]

> so less skilled coders just take the output of whatever the LLM spits and assume it's correct, later it turns out there is a bug. The person who submitted the code doesn't understand the code, because they didn't bother learning it just that it was mostly correct. Instead of discovering and fixing the source of the error they just go in and discover where the error is output and slap some more AI generated code over it to squash the bug.

Ever worked with offshored programmers (won't name country or companies)? The things I've seen. Getting code via email that doesn't compile. Then seeing they replied-all, cc'd some random employee not on the project and 2-3 managers about "getting the urgent fix". Having a teenager with a 3 months bootcamp, an AI and a few post-it with git commands would actually be an improvement over this.

> The issue is that the biggest thing AI brings is it allows you to generate lots of code, very fast. So we have more and more code, that is less and less understood by any human being.

I know guys whose whole contracting careers is re-writing (completely from scratch) huge codebase that were offshored. 10x reduction in line count is pretty much standard. I guess these AIs means their business model will keep working for the next decade!

[antonvs]

What you're describing will primarily happen at incompetent orgs without good development practices, code review, good management, good hiring, etc.

If anything, a tool that reveals and punishes incompetence is good for everyone. I'm not the world's most enthusiastic capitalist, but in this case, I think we can let the market sort it out.

[jmnicolas]

Good for you that you can choose only the best.

The rest of us are in the trenches making do despite the unending stream of spaghetti code that cruel gods have chosen to torment us with.