[redleggedfrog]

I'll share a new wrinkle that casts more shade on the coding LLMs.

We have a fair number of offshore resources that are used for dev. They developers are fully integrated into the team, are in all the stand-ups, and substitute for the usual role of junior programmers. They don't get the grunt-work shoveled on them, they get the same work as everyone else, they're just expected to not be as fast.

In 6 months 2 out of 4 of them been sacked, and surprise, not because we could replace their work with LLM output, but because their use of LLMs was so unrestrained and scattershot the pull requests they submitted had become nightmares. One thing mentioned in the article about unit test creation was something we saw as well. Perhaps this is partly due to working an existing code base where the LLM loses some of its advantage, and certainly some of it was cultural in that progress was thought more important than actual manageable code. The two sacked fellows where told, literally, from my own mouth, multiple times, "You cannot just ask Copilot to write you code, paste the entire thing into Visual Studio with no thought of what has changed, with the end goal of just compiling and meeting the single set of acceptance criteria on your story. You're breaking other things and introducing bugs." It went on deaf ears, and now they're gone. They were nice people, I didn't know how to get through to them, but they were convinced the LLMs were the way to go.

I use LLMs to help write code every day, and I wouldn't want to be without it, but I'm fairly surgical about it. Most of the time Copilot gives you a page of say, React code, or EF Core queries, you have to be really careful about anything you didn't explicitly ask for. Honestly, there is a time savings, but there is not a quality increase. The benefit is subverted by the time it takes to figure out how to ask correctly, the time to vet the output, and the time to fix the little tiny insidious bugs it can introduce.

So, don't go vibe coding and lose your job, is something to think about. I have to admit that it has worn me down meeting these interesting people from far-flung locations only to watch them flounder and get let go.

[vunderba]

I've been contracting for a fairly largish company (around 300 devs split across ~40 teams). The US based company was bought out by private equity about a year ago and many senior/lead engineers were forced out and contracted out to cheaper overseas labor.

The company has been relatively ambivalent about the usage of code assistant AI, but during PR reviews it has become very apparent that its seen widespread adoption among the outsourced dev teams purely because of code duplication. Our company has a fairly large number of repositories and bespoke libs for utility type functionality.

In the past, a programmer might have internally said to themselves, "There's no way that somebody hasn't already written this stupid function X or method Y", and they'd take a few minutes to search or reach out to see if it exists within an organization.

Instead during some of the recent code reviews, there has been a huge uptick in core functionality that is very obviously being spit out by the LLM. At best its just extra unnecessary code. At worst it will introduce new bugs since our custom functions often handle business domain specific edge cases that an LLM simply wouldn't know about.

[redleggedfrog]

Totally lines up with my experience as well. We also have the opposite problem of reams of code generated years ago from a period of unsupervised offshore work where we're slowly paying down the debt but the LLMs will attempt to use the old code for new work. Most of it is spaghetti UI code and nearly impossible to reuse effectively but the LLMs give it their best and we have prompt around it.

[lubujackson]

I think people miss that "vibe coding" is a senior engineering tool. You still have to architect the project. You still have to ensure security and performance are handled (if relevant for the task). You still have to envision edge cases and usage patterns. You just don't have to type it out.

Now non-programmers can fumble forward to a working demo. But junior engineers are walking around with a loaded weapon - if they are not learning from how AI solves a problem or using it like StackOverflow to answer specific questions, they are blowing up their own careers.

The future is we are all product engineers or domain experts. No one is going to want an army of React engineers in 2 years.

[MoonGhost]

BTW, Copilot is not the best at coding. With the quality LLM return exponentially grows. Bigger chunks, fewer bugs, less time checking. From my experience LLMs do not impress on complex algorithms and shine on small utilities. They can use libs I even don't want to learn about.

[redleggedfrog]

I haven't really noticed that myself. I go "LLM shopping" fairly frequently trying to find which one of the few I'm paying for gives the best result for the current problem. They all seem to have their shortfalls, although I will say Claude is better for Greenfield work.

[MoonGhost]

Well, probably you can try to go down to simpler models to get the idea. (they are almost useless) From my experience better model like Claude or o3 can do things that others simply cannot. At some complexity they start going circles making wrong decisions, forgetting things that are still in context window. But the thing is those complex tasks are usually the most interesting and important.