[cosmic_cheese]

Additionally, LLMs still don’t truly understand anything, which is why they flounder so badly with e.g. writing code for a programming language or framework that it hasn’t seen a large enough set of training data for. Humans on the other hand do understand and generalize shared knowledge well, which is why we’re much better at handling that type of scenario.

More specific to agents, humans can also figure out how to use tools on the fly (even in the absence of documentation) where LLMs need human-built MCPs. This is also a significant limiting factor.

[tkz1312]

I’ve found claude to be very helpful when both writing and debugging code written in a language i’m currently building. I just make sure to load the spec into its context first and that seems to be enough for it to get a general understanding.

[brabel]

Everyone criticizing AI for not "understanding" anything... yet, as you found, and many others have also shown before, explain something to them and they bloody well look like they do understand it. I am still in awe at what LLMs can do, TBH. Over the last few months, the main problem with them: of confidently making shit up, seems to be getting much less of a problem... it's still not solved, but if things keep improving I wouldn't be surprised they will have controls that ensure they stop doing that, and when that happens people will be able to trust what they say/write much more... and perhaps that will be a turning point when complaints like in this post will be hard to take seriously.

[cosmic_cheese]

The issue is that their “understanding” following an explanation is quite shallow. They often miss many connections and underlying principles that a human would grasp right away, needing to be spoon-fed these things to fill the gap.

That’s not to say they’re not useful in their current state. They are. However, I believe it’s becoming clear that there’s a hard ceiling to how capable LLMs in their current form can become and it’s going to take something radically different to break through.

[actinium226]

> controls that ensure they stop doing that

I'm not sure you can do that. As humans, we need to make things up in order to have theories to test. Like back in the day before Einstein when people thought that light traveled through an "aether" whose properties we needed to figure out how to measure, or today when we can't explain the mass imbalance of the universe so we create this concept called "dark matter."

Also, in my experience the problem has been getting worse, or at least not better. I asked Claude 3.7 some time ago how to restore a snapshot to an active database on AWS, and it cheerfully told me to go to the console and press the button. Except there is no button, because AWS docs specifically say you can't restore a snapshot to an active database.