[tibbar]

The non-determinism is one of the relevant features of this layer of abstraction! And one can learn to validate that the translation is being done properly. Some of the tools you have include writing extremely detailed specs, generating visualizations of the internals of the tool, or (perhaps) reading the code, though that becomes less feasible with volume.

Basically it turns out that code is full of incidental details and what you really want is to verify the important parts, while receiving a guarantee that the vast tail of incidentals is handled "reasonably."

[bitwize]

LLMs will still blithely ignore the specs and steering documents, apologize profusely for doing so after the fact, and tell you "I'll do better next time" which they might do once or twice. But after the context is cleared or a new session opened, the Dixie Flatline gets reset and it doesn't remember it screwed up, or that you told it not to.

This happened to a coworker of mine. Generally the response from one-shotted devs is a shrug of the shoulders and "wellp, them's the breaks! As long as it looks sensible from 10,000 feet up it's still a huge productivity win." But the devil, as they say, is in the details.

[lenkite]

Writing Specs that are thousands of lines wrong without formalism, type and correctness checking is a major PITA. Wish we had a..compiler that can check our code..err I mean spec.

[duskdozer]

Oh, are you still handwriting your specs? How Q2 2026. Handcrafted artisan specs are obsolete and you will be left behind in the dust if you still insist on them.

[weakfish]

Poe’s law displayed beautifully here

[Izkata]

In which direction? It wasn't long ago having the agent generate a plan before acting on it was a "best practice".

[bcrosby95]

Writing specs validates the translation is being done properly? Was this written by an LLM?

[tibbar]

Writing detailed specs is an important tool to help the LLM translate the idea in your head to code. The spec is an intermediate artifact.