[giantg2]

If anything, you have to understand code more now.

Before you (or your devs) could write code a couple different ways and understand it. Now you have to look a code generated by an agent that is not necessary writing code in the same way as the culture at your company. There might be a thousand different ways a feature gets written. You have to spend more time reviewing and thinking it about it in my opinion.

[dapangzi]

Made a similar comment.

It's great for tenured engineers, when we use it.

When juniors use LLM, because they don't have experience, it becomes a nightmare for tenured engineers, and we just end up "mopping the slop", as I tend to say.

I also have issue with how LLM do testing.

[taurath]

Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.

[dapangzi]

> you don’t have standards.

The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes.

Edit: Also LLM over mock tests and juniors trust that...

[taurath]

Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.

[dapangzi]

> Not very often

> testing

This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it.

I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic.

We have had multiple customer impacting events from code juniors committed without understanding it. Please read my top level comment in this post for context.

I genuinely hope you do not encounter issues due to your confidence in LLM, but again, my experience does not match yours.

Edit: Would also add that LLM is not good at determining line numbers in a code file, another flaw that causes a lot of confusion.

[giantg2]

I had a mid-level submit a PR implementing caching. I had to reject it multiple times. They were using Copilot and it couldn't implement it right and the developer couldn't understand it. Stuff like always retrieving from the API instead of the cache, or never storing the object in the cache after retrieving it.

They promoted that guy over me because he started closing more stories than me and faster after he started using Copilot. No wonder that team has 40% of its capacity used for rework and tech debt...

[taurath]

I haven’t run into that problem but I do also hold agents on a tight leash!

[giantg2]

I've never had an LLM create a robust, meaningful test file. I end up rewriting at least half of it.