[inciampati]

But we do manage to remember stuff that happened "quadrillions" of tokens (of sensory input) back. Sure, we are wrong a lot, but we have a context that goes way back. A LLM has a fruit fly's version of what we work with, although in some versions it has perfect recall in that context.

[seanthemon]

Or it can be said that we boil down our experience to a few millions of tokens, only a set of experiences, emotions and often wrong/made up memories.

Your comment assumes a human can recall all (potentially incorrect) memories and use those to make some judgement, but fact of the matter is we don't.

[FrustratedMonky]

Yeah, Humans do ton of aggregation and filtering. Thus you can't remember individual days of commuting, because it is so similar, the brain just lets it go.

I'd think something like this would be added to the models eventually. Of course, over-simplifying.