[CharlesW]

> It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control.

You invoke "engineering principles", but software engineers constantly trade in likelihoods, confidence intervals, and risk envelopes. Using LLMs is no different in that respect. It's not rocket science. It's manageable.

[th0ma5]

But the threshold between correct and incorrect inference is dependent on an intersection of the model and the document so far. That is not manageable by definition, I mean... It is a chaotic system.

[bredren]

Is this dissimilar to what the human brain produces? Are we not producing chaos controlled by wanting to give the right answer?

[th0ma5]

Yes it is very dissimilar. Life isn't a sum of the discrete inputs. I mean maybe it is at times but the context is several orders of magnitude greater, the inputs several orders of magnitude input, etc but the theory that it can be quantified like this is unproven let alone a good basis for an artificial system.

[skydhash]

> but software engineers constantly trade in likelihoods, confidence intervals, and risk envelopes

Software engineering is mostly about dealing with human limitations (both the writer of the code and its readers). SO you have principles like modularization and cohesion which is for the people working on the code, not the computer. We also have tests, which is an imperfect, but economical approach to ensure the correctness of the software. Every design decision can be justified or argued and the outcome can be predicted and weighted. You're not cajoling a model to get results. You take a decision and just do it.