[root_axis]

There's no evidence for it, nor any explanation for why it should be the case from a biological perspective. Tokens are an artifact of computer science that have no reason to exist inside humans. Human minds don't need a discrete dictionary of reality in order to model it.

Prior to LLMs, there was never any suggestion that thoughts work like autocomplete, but now people are working backwards from that conclusion based on metaphorical parallels.

[LiKao]

There actually was quite a lot of suggestion that thoughts work like autocomplete. A lot of it was just considered niche, e.g. because the mathematical formalisms were beyond what most psychologist or even cognitive scientists would deem usefull.

Predictive coding theory was formalized back around 2010 and traces it roots up to theories by Helmholtz from 1860.

Predictive coding theory postulates that our brains are just very strong prediction machines, with multiple layers of predictive machinery, each predicting the next.

[red75prime]

There are so many theories regarding human cognition that you can certainly find something that is close to "autocomplete". A Hopfield network, for example.

Roots of predictive coding theory extend back to 1860s.

Natalia Bekhtereva was writing about compact concept representations in the brain akin to tokens.

[root_axis]

> There are so many theories regarding human cognition that you can certainly find something that is close to "autocomplete"

Yes, you can draw interesting parallels between anything when you're motivated to do so. My point is that this isn't parsimonious reasoning, it's working backwards from a conclusion and searching for every opportunity to fit the available evidence into a narrative that supports it.

> Roots of predictive coding theory extend back to 1860s.

This is just another example of metaphorical parallels overstating meaningful connections. Just because next-token-prediction and predictive coding have the word "predict" in common doesn't mean the two are at all related in any practical sense.

[A4ET8a8uTh0_v2]

<< There's no evidence for it

Fascinating framing. What would you consider evidence here?