[01100011]

The arbitrary and non-deterministic nature of LLM workflows gives me full on ick. As an old embedded/systems guy I have always prioritized determinism and repeatability in my workflows.

But damn, agents are amazing and I'm enjoying being a "thought process designer". I'm not going back. Even if AI development stops today my career will never be the same.

[nk91]

I felt the same way about the non-determinism but realized it can be really beneficial to have a machine that can fairly reliably turn non-determinism into determinism.

I’m working on a tiny agent harness at home to learn and the process of taking human speech and turning it into agent tool calls that output something generally deterministic depending on how the tool is defined is so interesting.

One of the big takeaways is you really only have to rely on the non-determinism<->determinism translation layer once when you switch between the two domains. You can obviously rely on it more if you want, and that’s probably faster because determinism is hard, but you don’t need too do that.

[port11]

That sounds very cool. It’s sometimes baffling that LLMs can’t use tools reliably. Serena and Semble both require some arcane instructions to coerce Claude Code into compliance. Just stop trying to pipe nonsense commands into each other, man!

[nk91]

I think it makes sense when you dig into why that non-determinism conversion is so hard.

For voice related things you have a lot of turn of phrase scenarios that can make no sense unless you know. Phrasing like “Put Larry on the horn.” For someone familiar with old lingo for phone calls makes sense. For someone else they might think of a war horn, someone else a music class.

All of those are wildly different situations. It’s not hard to see how one oops between two non deterministic things can quickly go off the rails.

The fact we can get away with so much non-determinism->non-determinism recursion is frankly amazing when you realize how easy it is to imprecisely describe what it is you’re thinking.

[port11]

The vagary of speech and its meaning is surely hard to parse. But! How many ways must a model invent to run `tsc`?

    npx tsc
    bash tsc
    bash npx tsc
    npm run build
    …

I’m not an expert at all on the subject matter, but is it impossible to train a model that calls tools in a (quasi-)deterministic way?

[newterminus]

it has a random seed cant you just set it

[coldtea]

Wouldn't change much.

Would just make the answer to the same exact prompt X repeatedly the same.

It wouldn't change the fact that prompt X', functionaly indistinguisable from X, aside from small phrasing changes, can give a totally different looking answer.

[newterminus]

when i make a statement describing the world it will be a totally opposite answer to the prompt "synthesize and analyze the steelman that X" because you are asking it to create something new and then you are giving it room to counter signal and some space to escape into but often it doesnt even countersignal much or at all in this analysis phase

[ACCount37]

It's like working with humans.

Can't help but feel like a lot of people who are deep in IT made it there because they hated working with humans.