| Have a look at any mass manufacturing process. Raw materials in, lots of process steps in the middle with tolerances etc; some % yield of goods out. When you work with LLMs, the "raw materials" is its first attempt to answer your prompt. Mass manufacturing mirrors what we do with LLMs to knock their output into shape: grounding, automated quality checking tooling (eg. linters for code, grammar checkers for prose), and ultimately, as many turns as needdd to manually bang the output or work product into shape... You should never ship your AI's first draft - that'd be slop. But you also shouldn't have to repeat yourself to get its output into a form you can use. If you know you're always going to have to tell it to change one specific thing, see if you can automate a process that tells it to change that thing for you, so you don't have to. Do this enough times and you've created a deterministic outcome at least at some level. And you're out of that loop of getting it to that level. Tooling for this is limited today. You can provide instructions; but you're always at the mercy of labs to make models that follow instructions and at their mercy that they didn't put conflicting instructions in the system prompt. Better to put in things like linters and grammar checkers - even going as far as putting in a fact checking process or for legal stuff, a citator. Without these extras, AI output is no more real than some dream you had. |