[dazamarquez]

I use AI to write specific types of unit tests, that would be extremely tedious to write by hand, but are easy to verify for correctness. That aside, it's pretty much useless. Context windows are never big enough to encompass anything that isn't a toy project, and/or the costs build up fast, and/or the project is legacy with many obscure concurrently moving parts which the AI isn't able to correctly understand, and/or overall it takes significantly more time to get the AI to generate something passable and double check it than just doing it myself from the get go.

Rarely, I'm able to get the AI to generate function implementations for somewhat complex but self-contained tasks that I then copy-paste into the code base.

[missinglugnut]

My experience is very similar.

For greenfield side projects and self contained tasks LLMs deeply impress me. But my day job is maintaining messy legacy code which breaks because of weird interactions across a large codebase. LLMs are worse than useless for this. It takes a mental model of how different parts of the codebase interact to work successfully and they just don't do that.

People talk about automating code review but the bugs I worry about can't be understood by an LLM. I don't need more comments based on surface level patter recognition, I need someone who deeply understands the threading model of the app to point out the subtle race condition in my code.

Tests, however, are self-contained and lower stakes, so it can certainly save time there.

[aspaviento]

Legacy code, files with thousands of lines, lack of separation of concern... When I feed that to an LLM it says something like:

function bar() { // Add the rest of your code here }

And barely manages to grasp what's going on.

[sourdoughness]

Interesting. I treat VScode Copilot as a junior-ish pair programmer, and get really good results for function implementations. Walking it through the plan in smaller steps, noting that we’ll build up to the end state in advance ie. “first let’s implement attribute x, then we’ll add filtering for x later”, and explicitly using planning modes and prompts - these all allow me to go much faster, have good understanding of how the code works, and produce much higher quality (tests, documentation, commit messages) work.

I feel like, if a prompt for a function implementation doesn’t produce something reasonable, then it should be broken down further.

I don’t know how others define “vibe-coding”, but this feels like a lower-level approach. On the times I’ve tried automating more, letting the models run longer, I haven’t liked the results. I’m not interested in going more hands-free yet.