[brookst]

Re: less common, I was just saying it doesn’t take millions off examples like PlanetUML.

> what can you trust

I wasn’t clear enough here — you’re responding to DSLs as an interface from non-deterministic LLMs to deterministic external systems.

What I meant was using DSLs as intermediate checkpoints in multi-LLM processing. If you just flow natural language through 5 LLM calls, the last one may be getting something very different from what it’s prompt is designed for.

But if you make the DSL a contract for handoff, results are much more stable.

Perfect and deterministic? No, of course not. Just an improvement and mitigation. But it’s served me well.

[codegladiator]

its served me okay.

> Perfect and deterministic? No, of course not. Just an improvement and mitigation.

exactly, not reliable, as the article tries to portray

[brookst]

Interesting … do you see reliable == infallible? I don’t. But certainly all software is imperfect, and LLMs doubly so.

[codegladiator]

for me reliable = deterministically repeatable. if a llm has been able to do a task successfully and i ask it to do the same task again, i want the reliability that i get the same outcome (be if failure or success)

like if someone says "is this car reliable", i dont expect an infallible car. but if the cars third gear is 'you know sometimes it doesnt work', i wouldnt take that car out of city.

[samatman]

This seems too strong to me. I agree with reliable as "repeatable", it's the determinism which seems excess to purpose.

If I have a great meal every time I go to a restaurant, that chef is reliably good. I do not require that the tomatoes on the salad be of identical number or placement, or cut along the precise same bias. Nor would I count the sesame seeds on the bun.

Are LLMs reliable by that standard? Sometimes yes, for some tasks. The envelope of tasks for which they're reliable is continuing to grow.