Hacker News new | ask | show | jobs
by imtringued 72 days ago
It is hypothesized that the human brain uses predictive coding for obvious biological reasons such as energy efficiency (spiked error coding means only differences need to be transmitted) and biological plausibility (only local communication is permitted, meanwhile backpropagation is a global algorithm).

Transformers have a thing called a context window which doesn't really have a biological equivalent, since the brain has a fixed size and doesn't grow or shrink in response to the amount of information being processed.

LLMs consist of several layers that communicate at fixed points between the layers, whereas neurons can form feedback loops and communicate with any neighbour in any direction.

Humans do not consume or produce tokenized information. The brain controls the human body which is a biomechanical system. Spoken or written language is the result of controlling muscles via an internal model of the biomechanical system, not something that was designed via a software tokenizer that compresses character sequences.

The equivocation just doesn't seem appropriate. Try again in 2050.