Hacker News new | ask | show | jobs
by RaftPeople 50 days ago
> Animals are also next token/action generators

But for humans, the concept/thought/idea/action is formed first and then a sequence of tokens are generated to communicate that concept/thought/idea/action.

1 comments

And a lot of GPU cycles happen before next token prediction, what's your point?
The point was that next token generation for a human was based on constructing something that matches the thought that is held in the mind.

LLM's generate the next token based on a statistical relevance of trained data plus the previous tokens generated.

Under normal conditions, a human generating tokens would not diverge down a different path from the thought that they were trying to communicate. All of the words/tokens generated support the idea or thought being communicated.

LLM's frequently generate tokens that do not make sense, and mathematicians have shown that hallucinations can not be eliminated from the current model of LLM's.