[embedding-shape]

> The big problem is that a person making a mistake can be taught to not make that mistake again. That's also not foolproof but at least it works a lot of the times. AI are unteachable, if you have given them a good prompt and they do something wrong 90% of the time you are shit out of luck.

I feel like this line of thinking is kind of an unfair comparison. I'm not saying LLMs are magical beings that can suddenly learn by themselves after getting something wrong, but your "person making mistake then being corrected" assumes you do tell the person about the mistake and tell them to avoid doing the same mistake in the future, but for the "LLM making mistake" example you then intentionally avoid letting the prompt being changed in response to the mistake, which would be the "then being corrected" part on the LLM side of the comparison.

Similarly, if you just let a person make a mistake and don't let them know about the mistake, they might keep making that same mistake over and over again.

If you update how you use the LLM as you discover what mistakes it does, just like you'd correct a person, then you can use an LLM and also the LLM can "be taught to not make that mistake again".

[rowanG077]

I'm not against the prompt being changed, the point I was making is that an LLM is prone to the exact same mistakes even if you change the prompt. A trivial example is the very basic character counting mistake, I just asked chatgpt:

> How many p are in strawperry?

> There are 0 “p”s in strawperry.

And I can trigger the same mistake with various words even when adjusting the prompt many times. So I cannot teach chatgpt to correctly count characters.

[embedding-shape]

> the point I was making is that an LLM is prone to the exact same mistakes even if you change the prompt

That'd be strange, what specific models?

Your example of "changing the prompt" doesn't actually tell the model what to avoid though, you're just rephrasing the question? Again you're not doing an equal comparison for the human, because in that example I bet you'd tell the human what they messed up and how to do it better, why not include that in the prompt when you compare these two?

[1659447091]

> That'd be strange, what specific models?

Fable. (and other claude ones)

I have a clojure/babashka environment that I work in. My agents/claude files specifically explain this and how to work with it and what to use etc. Fable decided it didn't want to use bb/clj and ran python to do cli work in one project.

So I update the files in all projects to specify that python is strictly off limits/not to be used/never to run it/etc and created a preToolUse hook -- because I learned that just because an agents file says something doesn't mean it gets followed. Hence the hooks, fool me once ...

Later, I start work in another project with the hooks and updated files saying to use the established bb/cli tools and to absolutely never run/use python etc. What does it do on the first task? Tries to run python, gets hit with the preToolUse hook error and replies with something like -- oops my bad I tried to use python even though the instructions told me not too. Every once in awhile in new tasks I still see the hook error at some point but it doesn't apologize like it did before it just switches to proper tools after it can't run what it wants even though it's in the instructions and could have been avoided.

[unparagoned]

You probably should test your fake scenarios before making claims on how what llms do.

Why is it that almost every time someone makes a claim it turns out wrong when tested