[tossandthrow]

I regularly check in on using LLMs. But a key criteria for me is that an LLM needs to objectively make me more efficient, not subjectively.

Often I find myself cursing at the LLM for not understanding what I mean - which is expensive in lost time / cost of tokens.

It is easy to say: Then just don't use LLMs. But in reality, it is not too easy to break out of these loops of explaining, and it is extremely hard to assess when not to trust that the LLM will not be able to finish the task.

I also find that LLMs consistently don't follow guidelines. Eg. to never use coercions in TypeScript (It always gets in a rogue `as` somewhere) - to which I can not trust the output and needs to be extra vigilant reviewing.

I use LLMs for what they are good at. Sketching up a page in React/Tailwind, sketching up a small test suite - everything that can be deemed a translation task.

I don't use LLMs for tasks that are reasoning heavy: Data modelling, architecture, large complex refactors - things that require deep domain knowledge and reasoning.

[NumberCruncher]

> Often I find myself cursing at the LLM for not understanding what I mean...

Me too. But in all these cases, sooner or later, I realized I made a mistake not giving enough context and not building up the discussion carefully enough. And I was just rushing to the solution. In the agile world, one could say I gave the LLM not a well-defined story, but a one-liner. Who is to blame here?

I still remember training a junior hire who started off with:

“Sorry, I spent five days on this ticket. I thought it would only take two. Also, who’s going to do the QA?”

After 6 months or so, the same person was saying:

“I finished the project in three weeks. I estimated four. QA is done. Ready to go live.”

At that point, he was confident enough to own his work end-to-end, even shipping to production without someone else reviewing it. Interestingly, this colleague left two years ago, and I had to take over his codebase. It’s still running fine today, and I’ve spent maybe a single day maintaining it in the last two years.

Recently, I was talking with my manager about this. We agreed that building confidence and self-checking in a junior dev is very similar to how you need to work with LLMs.

Personally, whenever I generate code with an LLM, I check every line before committing. I still don’t trust it as much as the people I trained.

[tossandthrow]

> ... Who is to blame here?

That is not really relevant, is it? The LLM is not a human.

The question is whether it is still af efficient to use LLMs after spending huge amounts of time giving the context - or if it is just as efficient to write the code yourself.

> I still remember training a junior hire who started off with

Working with LLMs is not training junior developers - treating it as such is yet another resource sink.

[NumberCruncher]

I think we have to agree that we disagree. What works for you doesn't have to work for me and vice-versa.

[tossandthrow]

If you want to sit and throw tokens after thinking you are treating an LLM like a junior developer, all power to you.

Meanwhile, I think I will stick to get tangible performance benefits out if its usage.