[wg0]

We can build all the scaffolding around but I assure you that the LLMs aren't perfect rule following machines is the fundamental problem here and that would remain.

Give it a few more months and I'm sure you'll see some of what I see if not all.

I'm saying all the above having all sorts of systems tried and tested with AI leading me to say what I said.

[cortesoft]

I have been doing this for 6 months or so now, and I am not sure that even if you have a lot more experience than me that it would make your assessment more accurate, since that just means you have more experience with prior generations of the models. What I have experienced is that the AI has been getting better and better, and is making fewer and fewer mistakes.

Now, part of that is my advancements as well, as I learn how to specify my instructions to the AI and how to see in advance where the AI might have issues, but the advancements are also happening in the models themselves. They are just getting better, and rapidly.

The combination of getting better at steering the AI along with the AI itself getting better is leading me to the opposite conclusion you have. I have production systems that I wrote using spec-kit, that have been running in production for months, and have been doing spectacularly. I have been able to consistently add the new features that I need to, without losing any cohesion or adherence to the principals i have defined. Now, are there mistakes? Of course, but nothing that can't be caught and fixed, and not at a higher rate than traditional programming.

[Quarrel]

> LLMs aren't perfect rule following machines is the fundamental problem here

I kind of get what you're saying, but let us not pretend that SW engineers are perfect rule followers either.

Having a framework to work within, whether you are an LLM or a human, can be helpful.

[suttontom]

If someone regularly ignored critical instructions even though they were written down and had been told to follow them, that person would be fired.

[scottg489]

People are excused all the time for things because they are elevated in other areas. It's about their value as a whole and that's where we are with LLMs. They aren't perfect but they do plenty we can't which means they are worth using.

[saidnooneever]

i think it depend on your goals and also your preference / expectation how your experience with LLMs is. i dont mind if they hallucinate. even if i have mental model of code i wont write it myself perfectly either.

the only downside i see is getting out of practice, which is why for my passion projects i dont use it. work is just work and pressing 1 or 2 and having 'good enough' can be a fine way to get through the day. (lucky me i dont write production code ;D... goals...)

[albedoa]

> Give it a few more months

By that time, they will have realized immense value before seeing some of what you see. Sounds like an endorsement of spec-kit.