[suddenlybananas]

>The thing that pushes me over the line into ranting territory is that computer programmers, of all people, should know that computers do what you tell them to.

are you claiming LLMs function like computer program instructions? like they clearly don't operate like that at all.

[mordymoop]

They are closer to being deterministic machines that comply exactly with your instructions, for better or worse, than they are to magical pixies that guess what you must’ve actually meant. The implicit expectation demonstrated by many in the “loudly disappointed in LLMs” contingent seems to be that LLMs should just know what you meant, and then blame them for not correctly guessing it and delivering it.

I think LLMs have uncovered what we have always known in this industry: that people are, by default, bad at communicating their intent clearly and unambiguously.

If you express your intent to an LLM with sufficient clarity and disambiguation, it will rarely screw up. Often, we don’t have time to do this, and instead we aim for the sweet spot of sufficient but not exhaustive clarity. This can be fine if you are experienced with that particular LLM and you have a good feel for where its sweet spot actually is. If you miss that target, though, the LLM will not correctly infer your intended subtext. This is one of the things that requires experience. In fact, even the “same” LLM will change in its behavior and capabilities as it undergoes fine tuning. Sometimes it will even get worse at certain things.

All of this is to say, of course, you’re right that it’s not a compiler. But I think people fail in their application of LLMs for much the same reason that novice coders fail to get compilers to guess what they intended.

[Dylan16807]

> They are closer to being deterministic machines that comply exactly with your instructions, for better or worse, than they are to magical pixies that guess what you must’ve actually meant.

If those are your only two reference points, yes they're closer to the former.

But the biggest problem is how much "pixie that does something you neither wanted nor asked for" gets mixed in. And I think a lot of the complaints you're saying are about lack of mind reading are actually about that problem instead.

[dambi0]

What part of the comment makes you think they are claiming that?

[suddenlybananas]

The part I quoted? I don't really see how to interpret it any other way.

[dambi0]

Thanks, I’m not thinking clearly at all. I had it in my head that programming instruction are not the only way that a computer might do as it’s told. For example, if we delete a folder in a UI by accident we shouldn’t be surprised the folder is gone. But it doesn’t really quite fit the parent analogy. Sorry about that.