[fzeindl]

I was originally sceptical of LLMs and am far from the „agents will magically fix our future“-crowd, but sentences like these trip me up:

> „But pattern‑matching is not system understanding, and plausibility is not correctness.“

Why not? Who says that? Who proved that system understanding is not just more complex pattern matching?

> „LLMs predict tokens, not consequences“

Same here. LLMs output tokens but who says that they don’t form some internal group of token-predicting tensors that move together and constitute the internal model of a „consequence“? It is like saying humans don’t have thoughts, they just have electrical impulses moving their tongues.

I too think that LLMs seem to be a very specific form of intelligence, maybe resembling the parts of our brain that do language-processing, but it is a fact that they at least fake intelligence very convincingly. And that we actually don’t know how they do it.

[pjm331]

> > "But pattern‑matching is not system understanding, and plausibility is not correctness."

> Why not? Who says that? Who proved that system understanding is not just more complex pattern matching?

I'm not in the camp of "system understanding is just more complex pattern matching"

but I am absolutely in the camp of "there are many tasks where pattern matching is just as effective as actual understanding"

[harperlee]

More strongly, if the pattern matching of a phenomenon totally / perfectly models the phenomenon, and you end up with a perfect model of the phenomenon, that enables you do do causal prediction, how can you NOT call it understanding? What more is there?

[fzeindl]

> but I am absolutely in the camp of "there are many tasks where pattern matching is just as effective as actual understanding“

What if „being effective at something with pattern matching but not understanding it“ just means that you have identified only 90% of patterns and keep failing to learn the rest for whatever reason.

[soco]

Aren't we humans functioning in the same way, failing for 10% (take a random number) whatever we learn because we can forget, or be tired, or distracted? And what is the practical effect of "actual understanding" other than actually getting the 90% right (or more, or less, whatever)? I cannot tell what's inside my neighbor's head so for all practical matters they could be an AI, so why should I care whether the AI has a real understanding (good luck proving that) or not? I only care whether they take away enough jobs (mine included) that I cannot life a peaceful life anymore because it sends me foraging for roots or I must defend my roots parcel against hungry foragers. And for AI to achieve that it definitely doesn't need "actual understanding" just following some less or better formulated goals and having the right tools under their "hands".

What I want to say is, yeah fascinating topic about real understanding, but I think we have more pressing issues.

[locknitpicker]

> Why not? Who says that? Who proved that system understanding is not just more complex pattern matching?

Yes indeed. That's a perplexing statement considering that a central concept or software engineering is architecture patterns.

[thesz]

  > central concept or software engineering is architecture patterns.

Both RUP and PSP/TSP do stand on the ground of defect prevention. All sorts of defects, from incorrect sets of requirements to memory corruption.

Architecture patterns can be of help in that regard and they also can be very error-prone, as right now I am in the process of removing a bug introduced through misunderstanding of one rather old singleton.

[locknitpicker]

> Both RUP and PSP/TSP do stand on the ground of defect prevention. All sorts of defects, from incorrect sets of requirements to memory corruption.

Was this comment generated by an hallucinating agent? It reads like poorly pieced together word soup.

[thesz]

English is not my native language.

PSP/TSP (Personal and Team Software Processes) and RUP (Rational's Uniform Process, to a lesser extent) are very valuable approaches to software engineering. Please, read about them, they are very interesting.

[locknitpicker]

> English is not my native language.

That's not it. It's the random word soup. PSP/TSP has absolutely nothing to do with the discussion.

[thesz]

PSP/TSP have everything to do with the discussion.

[justincormack]

The whole post is written by AI anyway, so its not worth engaging with.

[amelius]

> Why not? Who says that? Who proved that system understanding is not just more complex pattern matching?

I think the naysayers already decided that the burden of proof is on the other side.

[coldtea]

That's how the burden of proof in such claims works in science.

They naysayers didn't come around claiming they invented a form of intelligence in a program, AI companies/advocates did. Burden on proof is on them.

[pjc50]

That is the traditional "null hypothesis", yes.

[ertgbnm]

The null hypothesis isn't just the opposite of whatever your opposition believes.

For LLMs the null hypothesis would be that there is no relationship between the input and output tokens. Something that is so obviously not true that it's not even worth calculating the number of sigmas away from the null hypothesis that LLMs are.

So clearly we discarded the null hypothesis sometime in 2017. Now we have a system that is really really good at pattern matching and seems to understand consequences. Is that "seeming" just a ruse or does it really understand stuff? A proper scientists would look at that evidence and put forward the hypothesis that maybe it really does understand stuff and begin working on experiments that would disprove that alternative hypothesis, moving forward with the assumption that the hypothesis is true until disproven or a better hypothesis is proposed that explains previous evidence more accurately. Naysayers saying "you haven't proven that pattern matching becomes understanding to my satisfaction" is not a rebuttal. They need an alternative hypothesis that can make predications that better fit the model and can be tested.

The only rebuttals I've heard are "AI can't actually understand stuff and therefore can't do X" which is a testable hypothesis at least. But Invariably AI eventually does X, just in a different way than anyone really expected.