[anonzzzies]

Yeah, that seems just unclear language and because it's trained on human language, 'is' does not equal 'equals'. Using 'equals' will help.

[eloisant]

That's the whole problem of LLM: they work only on human language.

Even before computers we created formal languages (mathematics, logic equations) precisely because human language is too often ambiguous.

[Spivak]

Don't you lump math in there, math is 99% human language. The symbol pushing you learned in HS is just advanced arithmetic. Math is more like legalese with some very loose additional notation than a formal language.

[lkirkwood]

Can you expand on this? Notation like "=" can be written using language but we define an exact meaning for the operator regardless, unlike language.

[JieJie]

I suppose that begs the question, what if we trained an LLM only on examples of formal languages?

[DonaldFisk]

In the particular cases being discussed, there's no ambiguity: "is a" means "member of" and "is the" means equals.

[dragonwriter]

> In the particular cases being discussed, there's no ambiguity: "is a" means "member of" and "is the" means equals.

Yes, and fitting just those cases would result in a model that handled other cases incorrectly, because idioms inconsistent with that rule exist. (“Jodie is the bomb” has a meaning distinct from the individual words taken separately which is not stating a reflexive equivalency, for instance.)