[bartread]

Yeah, I would say it's pretty variable, and it depends on what you mean by the word write.

I've recently joined a startup whose stack is Ruby on Rails + PostgreSQL. Whilst I've used PostgreSQL, and am extremely familiar with relational databases (especially SQL Server), I've never been a Rubyist - never written a line of Ruby until very recently in fact - and certainly don't know Rails, although the MVC architecture and the way projects are structured feels very comfortable.

We have what I'll describe as a prototype that I am in the process of reworking into a production app by fixing bugs, and making some pretty substantial functional improvements.

I would say, out of the gate, 90%+ of the code I'm merging is initially written by an LLM for which I'm writing prompts... because I don't know Ruby or Rails (although I'm picking them up fast), and rather than scratch my head and spend a lot of time going down a Google and Stackoverflow black hole, it's just easier to tell the LLM what I want. But, of course, I tell it what I want like the software engineer I am, so I keep it on a short leash where everything is quite tightly specified, including what I'm looking for in terms of structure and architectural concerns.

Then the code is fettled by me to a greater or lesser extent. Then I push and PR, and let Copilot review the code. Any good suggestions it makes I usually allow it to either commit directly or raise a PR for. I will often ask it to write automated tests for me. Once it's PRed everything, I then both review and test its code and, if it's good, merge into my PR, before running through our pipeline and merging everything.

Is this quicker?

Hmm.

It might not be quicker than an experienced Rails developer would make progress, but it's certainly a lot quicker than I - a very inexperienced Rails developer - would make progress unaided, and that's quite an important value-add in itself.

But yeah, if you look at it from a certain perspective, an LLM writes 90% of my code, but the reality is rather more nuanced, and so it's probably more like 50 - 70% that remains that way after I've got my grubby mitts on it.

[WD-42]

This is exactly how I use AI as well in codebases and languages I’m not familiar with.

I’m a bit concerned we might be losing something without the google and stack overflow rabbit holes, and that’s the context surrounding the answer. Without digging through docs you don’t see what else is there. Without the comments on the SO answer you might miss some caveats.

So while I’m faster than I would have been, I can’t help but wonder if I’m actually stunting my learning curve and might end up slower in the long term.

[bartread]

Yeah, I had that concern as well, but I tend to find that both ChatGPT and Claude do a pretty good job of explaining the issues around the solutions I'm using them for, so I think - maybe not all - but a good portion of that learning is still happening. No doubt this is a result of the way I prompt them, because it does involve a lot of discussion, iteration, and back and forth.

Of course, you do have to know what you're doing well enough to spot a hallucination, and those do occur but, I mean, how often do you go down a blind alley or rethink your approach with coding, particularly with a new platform or framework, anyway? It doesn't feel like I'm wasting a huge amount of time here.

[captain_coffee]

So let me get this straight - you vibe code, make what you consider as necessary changes to the LLM-generated code, create PRs that get to be reviewed by another AI tool (Copilot), potentially make changes based on Copilot's suggestions and at the end, when you are satisfied with that particular PR you merge it yourself without having any other human reviewing it and then continue to the next PR.

Did I get that right or did I miss anything?

[bartread]

Not quite: remember I said I fettled the code myself? This might involve rewriting, refactoring, reorganising, and I'll make changes to Copilot's PRs as well, but still the percentage of code written by LLM remains what I'd consider to be very high.

I've been an engineer for more than 25 years so I know how to architect an application from the highest level down to the fine details, and I can obviously code in a variety of languages. Even with Ruby and Rails I know it well enough to read and understand the code already. Plus I obviously know HTML and JavaScript, and have used various CSS frameworks and overlays. (Though the truth is I basically hate fussing with CSS [and its various proxies], and I'm very far indeed from the king of UX or design anyway, so having an LLM to assist with that is a godsend.)

The strict definition of vibe coding, as far as I've understood it, is that you don't touch or even really look at the code at all, and make all modifications, do debugging, etc., via prompting the LLM or using an agent. Vibe coding tends to break down in critical areas like security, or when the contours and wrinkles of the domain become too complex, so I'd never trust that approach for anything substantial or where security is a critical concern. For most organisations the data that I'm working with in our app is going to be second only to financial information in terms of sensitivity, and it's a complex domain, so the problems we're solving are simply not amenable to a pure vibe-coding approach.

Going back to the original point on how much code LLMs are writing: with other languages that I'm more familiar with, where it might be quicker for me to write the code to do what I need myself rather than write a spec for the LLM, and modify afterwards, you would probably see quite a different picture.

[captain_coffee]

Yes OK, but that was not the point that I was trying to make.

You are developing in a language + framework that you are not familiar with, without having any human feedback in the process at all - see where I am going with this?

Even with a quarter of a decade of experience you are still in what can basically be reffered to as uncharted territory.

[bartread]

Define what you mean by uncharted territory?

- I've never used a new language or framework before? Nope

- I've never used both a new language and a new framework at the same time? Nope

- I've never tried to do real work in a new language and new framework that I'm not particularly familiar with? Nope

- I've never shipped work I've done in a new langauge and new framework that I'm not particularly familiar with? Nope

What specifically do you think is so magic about Ruby and/or Rails that you think I won't have encountered something similar in some other form - in some other language or framework - before?

And do you not think that the reviewing and changes that I am making to the code, the prompting I'm doing, the decisions I'm making about which changes to integrate and which to ignore, etc., as a human being, count as feedback?

At the end of the day I'm building a web app, and soon enough an API, and the concerns in building this web app and API are exactly the same as the concerns I've had in building other web apps and APIs.

The only difference is the language and the framework, and I've learned new languages and frameworks before. And, although I'm using an LLM to help, I've been doing that for the past 3 years using different stacks and the only notable difference is that LLMs are quite a lot better at helping to develop software than they were 3 years ago.

So how is this uncharted territory?

---

And, as an aside, not an issue you've raised but one I commonly see on HN: people moaning about job ads demanding X years of experience in this or that framework or language (with the parody being that the desired framework or language has only existed for X years, or maybe not even that long), and how this isn't really relevant to whether an experienced software engineer can do the advertised job or not. I happen to agree with that perspective but here's the thing: you can't have it both ways. Either that's true or it's not true, and I'm operating (as I have done before and as you can probably tell) on the basis that it is true.

[AstroBen]

This seems like a really short-sighted view. 6 months from now you'll be much more inexperienced than if you just went through the initial struggle (with an LLM's help!)

[bartread]

I'm not sure what you mean by this. As I mentioned in another comment, both ChatGPT and Claude to a pretty good job of explaining around the solution we're working on - because of the way I'm prompting them, no doubt - and it's not as if I haven't also worked through both Ruby and Rails tutorials.

What I'd observe is that it's simply a different way of learning, but to me it seems like it's working at least decently well, and it means that I'm able to devote more time to thinking about and working on our roadmap and higher level issues without completely screwing my work/life balance - and, critically, time with family because the kids are growing up fast.

[WD-42]

What does the struggle with an LLMs help look like?

[AstroBen]

"Explain this to me" until you're able to complete the task, instead of "do this for me"

[WD-42]

If you break down the problem to small enough chunks, these basically become the same thing. The hardest part of new languages is the syntax and new APIs, so you end up getting the code anyway.