[shkkmo]

Maps are useful, but they don't understand the geography they describe. LLMs are maps of semantic structures and as such, can absolutely be useful without having an understanding of that which they map.

If LLMs were capable of understanding, they wouldn't be so easy to trick on novel problems.

[logicallee]

> If LLMs were capable of understanding, they wouldn't be so easy to trick on novel problems.

Got it, so an LLM only understands my words if it has full mastery of every new problem domain within a few thousand milliseconds of the first time the problem has been posed in the history of the world.

Thanks for letting me know what it means to understand words, here I was thinking it meant translating them to the concepts the speaker intended.

Neat party trick to have a perfect map of all semantic structures and use it to trick users to get what they want through simple natural-language conversation, all without understanding the language at all.

[shkkmo]

> Got it, so an LLM only understands my words if it has full mastery of every new problem domain within a few thousand milliseconds of the first time the problem has been posed in the history of the world.

That's not what I said. Please try to have a good faith discussion. Sarcastically misrepresenting what I said does not contribute to a healthy discussion.

There have been plenty of examples of taking simple, easy, problems, and then presenting them in a novel way that doesn't occure in the training material, and having the LLM get the answer wrong.

[logicallee]

Sounds like you want the LLM to get the answer right in all simple, easy cases before you will say it understands words. I hate to break it to you but people do not meet that standard either and misunderstand each other plenty. For three million paying customers, ChatGPT understands their questions well enough and they are happy to pay more than for any other widespread Internet service for the chance to ask it questions in natural language, and even though there is a free tier available with high amounts of free usage.

It is as though you said a dog couldn't really play chess if it plays legal moves all day every day from any position and for millions of people, but sometimes fails to see obvious mates in one in novel positions that never occur in the real world.

You're entitled to your own standard of what it means to understand words but for millions of people it's doing great at it.

[shkkmo]

> I hate to break it to you but people do not meet that standard either and misunderstand each other plenty

Sure, and there are ways to tell when people don't understand the words they use.

One of the ways to check how well people understand a word or concept is to ask them a question they haven't seen the answer for.

It is the difference in performance on novel tasks that allows us to separate understanding from memorization in both people and computer models.

The confusing thing here is that these LLMs are capable of memorization at a scale that makes the lack of understanding less immediately apparent.

> You're entitled to your own standard of what it means to understand words but for millions of people it's doing great at it.

It's not mine, the distinction I am drawing is widespread and common knowledge. You see it throughout education and pedagogy.

> It is as though you said a dog couldn't really play chess if it plays legal moves all day every day from any position and for millions of people, but sometimes fails to see obvious mates in one in novel positions that never occur in the real world.

While I would say chess engines can play chess, I would not say the chess engines understands chess. Conflating utility with understanding simply serves to erase an important distinction.

I would say that LLMs can talk and listen. And perhaps even that it understand how people use language. Indeed, as you say, millions people show this every day. I would however not say that LLMs understand what they are saying or hearing. The words are themselves meaningless to the LLM beyond their use in matching memorized patterns.

Edit: Let me qualify my claims a little further. There may indeed be some words that are understood by some LLMs, but it seems pretty clear there are definitely some important ones that aren't. Given the scale of memorized material, demonstrating understanding is hard but assuming it is not safe.

[arolihas]

Some of us care about actual understanding and intelligence. Other people just want something useful enough that can mimic it. I don't know why he feels the need to be an ass about it though.

[crabmusket]

> Maps are useful, but they don't understand the geography they describe. LLMs are maps of semantic structures and as such, can absolutely be useful without having an understanding of that which they map.

That's a really interesting analogy I've never heard before! That's going to stick in my head right alongside Simon Willison's "calculator for words".