[skydhash]

I forgot where I read it, but the reason that natural languages works so well for communication is because the terms are labels for categories instead of identifiers. You can concatenate enough to refer to a singleton, but for the person in front, it can be many items or an empty set. Some labels may even be nonexistent in their context

So when we want deterministic process, we invent a set of labels where each is a singleton. Alongside them is a set of rules that specify how to describe their transformation. Then we invented machines that can interpret those instructions. The main advantage was that we know the possible outputs (assuming a good reliability) before we even have to act.

LLMs don't work so well in that regard, as while they have a perfect embedding of textual grammar rules, they don't have a good representation for what those labels refers to. All they have are relations between labels and how likely are they used together. But not what are the sets that those labels refer to and how the items in those sets interact.

[akavi]

> All they have are relations between labels and how likely are they used together. But not what are the sets that those labels refer to and how the items in those sets interact.

Why would "membership in a set" not show up as a relationship between the items and the set?

In fact, it's not obvious to me that there's any semantic meaning not contained in the relationship between labels.

[skydhash]

Because the training data does not have it. So we have the label "rock" which intersect with some other label like "hard" and "earth". But the item itself have more attributes that we don't bother assigning label to them. Instead, we just experience them. So the label get linked to some qualia. We can assume that there's a collective intersection of the qualia that the label "rock" refers to.

LLMs don't have access to these hidden attributes (think how to describe "blue" to someone born blind). They may understand that color is a property of object, or that "black" is the color you wear for funerals in some locations. But ask them how to describe the color of a specific object and the output is almost guaranteed to be wrong. Unless they are in a funeral in the above location, so he can predict that most people wear black. But it's a guess, not an informed answer.

[bwfan123]

This is a nice explanation of language, and the world-model that the language is intended to depict. If I understand you correctly, formalism is a kind of language where the world-model (ie, items and actions in the world depicted by the language) leaves no room for doubt.

[skydhash]

Pretty much and if we take programming languages, where inputs and RNG are not specified by the formal grammar of that language, the programmer needs to split them between good values and bad values. And ideally, halt the program when it detects the latter as the result would be nonsense.

So a program is a more restrictive version of the programming languages, which itself is a more restrictive version of a computer. But the tools to specify those restrictions are not perfect as speed and intuitiveness would suffer greatly (haskell vs python).