[DevarshRanpara]

Stop for one second and ask yourself a simple question. Where do your words come from?

When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. The word is never the start. The word is just the skin. The idea, the consciousness, is the thing sitting under it.

[vitamark]

"For an LLM, it is exactly the opposite".

Is it really? LLMs don't have words inside, for the most part they operate by applying transformations to a vector that does not contain any words at all. Words that come out of an LLM are just what sampler gives us by looking at the vector that is the result of those transformations.

Does this vector contain a world model? Some form of thoughts or reasoning to arrive at the result? That's an open question really.

Sure, rerunning that whole process for each token might not be the best solution, although that's an open question too. But saying that LLMs operate on words first is too big of an oversimplification

[rcxdude]

>Does this vector contain a world model? Some form of thoughts or reasoning to arrive at the result? That's an open question really.

It is, but it has been explored in various forms. Anthropic have some interesting papers where they can map out concepts to some extent as they exist inside the vectors of the LLM, and they can see e.g. that vectors relating to some concept can appear there some iterations before the LLM emits the tokens that spell it out. (though also, quite frankly, this shouldn't be all that surprising. If they were just picking the next token with no representation of what might follow it the result would be more like the 'one-word-at-a-time' party game than coherent text. They used poetry with a rhyming scheme as an example because it's something where you need to have some idea of what might rhyme with the line before and aim at it at the start of the next line in order to have any degree of success).

[HelloUsername]

There's plenty of people who speak before they think

[next_xibalba]

> Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it

There are enough neuroscience experiments demonstrating how we create post hoc explanations of our actions that I wouldn't trust this intuition prima facie.

[jcims]

I've noticed that more and more of my mental attention while speaking is observing the words that come out of my mouth rather than 'generating' them. I split time between rough pathfinding in my mind to fit a conceptual framework then kind of sit and listen as it comes out to think about how they sound. If I had to guess this happens on a 1-2 sentence chunk at a time.

[bluefirebrand]

The fact that we're even capable of doing a study like that and introspecting is probably a good indication of our cognitive abilities as humans

Consider someone designing a skyscraper. They're thinking about material strengths and such while they draft it, they aren't just rationalizing their choice later.

[mattas]

Word is just the skin is the load-bearing idea. The real unlock, if you will.

[pigeonwarz]

My favourite way of thinking about this is by going back to the End Poem at the end of Minecraft. The fact that words are an "interface", "very flexible" and "less terrifying than staring at the reality beyond the screen".

Maybe that reality is the true understanding of the neurobiology that defines our thoughts. To be reminded that the magic of experience can be reduced to neuronic hallucination is, frankly, horrifying. Maybe we dislike LLMs because we see their vectors and numbers as crude reminders of our own intellectual banality.

Or maybe I'm just feeling hungry and should really go eat dinner.

[epihelix]

I don't know about you, but I think with words all the time when reasoning through complex ideas. Where do you think the phrase "thinking out loud" comes from?

[yieldcrv]

We’re stochastic parrots most of the time, with a varying ability to reason but opting for pattern conformity usually

[hallway_monitor]

What is reasoning except applying another pattern on top of existing thought? Personally it seems like I and everyone else is simply pattern matching, albeit at a higher level than current LLMs. There is no difference in the process as far as I can tell, just different inputs.

[danielmarkbruce]

This is getting voted down because people just hate the idea that it's true.

[bluefirebrand]

No it's down voted because we didn't stochastic parrot ourselves into landing on the moon and building the internet and modern highways and skyscrapers

Dismissing humanity that was is absurd

[yieldcrv]

LLMs with reality interfaces (aka a body) can do those things, add world models and they'll do all those things better in our world, the first time and better than us. because the failures would have happened inside the world model

[bluefirebrand]

The point is that they cannot do that to themselves. They cannot decide to add world models and build exoskeletons and make that happen without our intervention

[yieldcrv]

Stochastically parroting a moving goal post that wasn’t presented as a point at all earlier, imo

I can play devils advocate too. The first agents running LLMs and other models that get a capable exoskeleton can do the same for the other LLMs that will be tasked to execute the guidance, agentically

[danielmarkbruce]

Yeah, we did. It took humans 100,000 or so years to land on the moon - we aren't so clever.

[enaaem]

In some meditation practices, the task is to observe the stochastic parrot in your head. You will soon realise that you cannot control this parrot. If you cannot control your thoughts, could it be that you are not your thoughts? And if you are not your thoughts--the word stream in your head--what are you then?