[michaelrpeskin]

In my experience, I'm using LLMs as my abstraction to "junior engineer". A junior engineer isn't deterministic either. I find that if you treat the LLM output like a person's output, you're good. Or at least in my projects, it's been very successful. I don't have it generate more code than I can review, or if I give it a snippet to help me fix it, if it ends up re-writing it like an ambitious engineer would do, I tell it to start over and make minimal changes.

I guess I'm not spun up about the determinism because I've been working at the "treat it like a person" level more than the "treat it like a compiler" level.

To me, it's really like an engineer who knows the docs and had a good memory rather than infallable code generator.

I work at a small company, so we don't have tons of processes in place, but I imagine that if you already had huge "standards" docs that engineers need to follow, then giving the LLM those standards would make things even better.

[skydhash]

The thing is you can quickly teach a Junior how to respect a specification contract, so that with very minimal oversight, you get the wanted implementation. And after a few years (or months), the communication overhead get shorter. What would have been multiple rounds of meetings and review sessions are a short email and one or two demos.

[QuercusMax]

What I've been learning as a 20% "harness engineer" is that in order to get the models to "learn" you need to add both documentation and static checks, as well as often custom skills. My main project at work has issues where the AI will often get super confused and step on itself trying to run tests - so the answer is writing better docs (AGENTS.md) and providing deterministic tools to work with the projects.

Large software projects (I'm thinking google3) often have large amounts of both of those things, as they're always getting new developers joining.