[benwr]

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea?

In case anyone is wondering: yes obviously even the dumbest current models correctly answer, given this prompt verbatim, that it's because of lunar new year.

[fwlr]

No, the driving through Chinatown question is a question for self-driving cars. It is not a question for LLMs. There is some other question for LLMs, and the author is using the ancient technique of analogy to get you to think about that question.

[benwr]

OP:

> Now, ask yourself what it's going to take for a car to know this. It's not going to be some specialized set of driving instructions. It's going to require a holistic view of, well, everything, and I will repeat my feeling that it will undoubtedly end up with a sense of self as a result.

Whatever it is that it would take, is demonstrably present in LLMs. The point I'm trying to make here is that the author seems not to have connected this fact to their assertion that current LLMs are coked up parrots.

And yeah the author is correct that the systems have some rudimentary sense of self! It's a confusing situation and I'm not personally thrilled about it! But things are changing quickly, and it's especially important to be paying attention to what's actually true rather than assuming the things are what you saw when you used one for five minutes in 2022.

[dreambuffer]

She's not wrong at all about her rhetorical implication (being, some information requires more than simple systems can provide), but she has concluded incorrectly that a car which does not know how to avoid a busy route is useless technology, or that the problem can only be solved by inventing life.