There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism. Only the former can be statistically modeled to characterize and work around the noise with any reliability. If you measure hallucination rate for one prompt it tells you nothing about hallucination rate for another prompt.
> There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism
I think I agree, but could you expand?
My point is humans have made progress on understanding systems without clean determinism. Genetics and cosmology is technically deterministic, practically speaking, but I don’t think that’s accessible to pre-modern societies. Instead, trends and tendencies were observed, acted and capitalized on.
One big difference is the former doesn't "know" it's being experimented on. Gregor Mendel was able to grow large numbers of pea plants and identify patterns in their inheritance because the non-determinism followed consistent statistical laws. A plant can't suddenly decide to follow different laws of genetics because it saw a biologist.
On the other hand, an LLM "knows" what a benchmark is, and is capable of detecting benchmark-like scenarios. A prompt that appears reliable in testing may fail unexpected in the real world. In this way, an LLM is like a stock market (also composed of intelligent agents): the act of experimenting on it changes its properties. Such systems are never reliably predictable.
In engineering you have tolerances to deal with non-determinism. ”Within these bounds” and ”given these assumptions then…” is the foundation of building something _on top of_ those things. LLMs are the same, it relies on heavily exact Turing machines as input but its output is entirely unstable. Even if you can get determinism it will never get anything resembling ”bounds” out of the box. That makes it a poor foundation for building on top of. Ie it’s not a screwdriver, it’s the monkey who’s holding it.
I do not understand the need to argue that monkeys are better than screwdrivers at screwing. Just let the monkey be the best version of a monkey.
I would argue that management is a better discipline to pull from for employing LLMs; It is better equipped to deal with non-determinism and going completely off the rails.
So now instead of improving the tools of biology so we can actually understand it deeply - we increase complexity of IT so we have to rely on muddy, side-effecty tools of biology to try to infer some of the properties of the systems we made. That's depressing.
I feel crushing sadness when I think about how humans made these wonderful deterministic machines that we can understand and manipulate to do our bidding. Within a scant few generations, though, we've managed to effectively turn them into "magic boxes" that, soon, we'll only be able to poke and prod at when they don't do what we want.
I think it's safe to say that computing should be mostly deterministic. I know compilers use heuristics that are stochastic, but come on. Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
> Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
Computing has never really been deterministic all the way down. Storage gets corrupted, RAM has soft errors, networks drop/reorder packets, schedulers race, caches go stale, distributed systems partition, query planners change plans, ...
Obviously those become tolerable because we've developed layers of contracts and understanding the bounds around them. Error correcting codes, checksums, retries, consensus, idempotency, false-positive rates, SLAs, etc.
If the abstraction is “delegate this task to a junior engineer, analyst, lawyer, designer, or support rep,” you dont expect deterministic behavior. There's review, constraints, escalation, checklists, tests, and accountability.