[Nevermark]

I am using syntax in a general form to mean patterns.

We are talking about LLMs and the debate seems to be around whether learning about non-verbal concepts through verbal patterns (i.e. syntax that includes all the rules of word use, including constraints reflecting relations between words meaning, but not communication any of that meaning in more direct ways) constitutes semantic understanding or not.

In the end, all the meaning we have is constructed from the patterns our senses relay to us. We construct meaning from those patterns.

I.e. LLMs may or may not “understand” as well or deeply as we do. But what they are doing is in the same direction.

[meroes]

> In the end, all the meaning we have is constructed from the patterns our senses relay to us. We construct meaning from those patterns.

Appears quite bold. What sense-relays inform us about infinity or other mathematical concepts that don't exist physically? Is math-sense its own sense that pulls from something extra-physical?

Doesn't this also go against Chomsky's work, the poverty of stimulus. That it's the recursive nature of language that provides so much linguistic meaning and ability, not sense data, which would be insufficient?

[kevinventullo]

What sense-relays inform us about infinity…

A waterfall that never seems to run dry. The ocean. Outer space. Time.

I think infinitude, i.e. a property of something which never ends, is a simplifying abstraction for the many things we come across in reality for which we can’t know where it ends or, in a specific context, we don’t care.

[Nevermark]

> Appears quite bold. What sense-relays inform us about infinity or other mathematical concepts that don't exist physically?

A great point. A fantastic question.

My guess is:

1. We learn useful patterns that are not explicitly in our environment, but are good simpler approximations to work with.

Some of these patterns only mean something in a given context, or are statistical happenstance.

But some of them are actual or approximate abstractions, potentially applicable to many other things.

2. Then we reason about these patterns.

Sometimes we create new patterns that reveal deeper insights about our environment.

Sometimes we create nonsense, which is either obviously nonsense, fools those who don't reason carefully (i.e. bullshit). And some nonsense is so psychologically attractive that it helps some of us pose and believe we are special and connected to higher planes.

And sometimes we create patterns that offer deeper insights into patterns themselves. I.e. abstractions, like counting numbers, arithmetic, logic, and infinity.

--

It is worth considering, that the progression of abstractions, from unary counting, to more scalable number notations, zero as a number, negative numbers, etc. took a couple hundred thousands years to get going. But once we got going, every small new abstraction helped progress compound faster and faster.

At the level of abstract thinking, I view humans as intelligent as a species, not as individuals. Even the greatest minds, a very small proportion of us, had to stand on innumerable inherited abstractions to make significant progress.

Now many people contribute to new abstractions, but we have inherited powerful abstractions about abstractions to work with.

Erase all that accumulated knowledge for a new generation of humans, and very few would make much or any accumulated progress in explicit abstractions for a very long time.