Yes
When I have a conversation with GPT4, it keeps the history, and the entire history is a prompt for the next prompt.
I would say, not totally unlike a human memory.
just GPT4 is limited to 8000 characters or something.
Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.
But we do manage to remember stuff that happened "quadrillions" of tokens (of sensory input) back. Sure, we are wrong a lot, but we have a context that goes way back. A LLM has a fruit fly's version of what we work with, although in some versions it has perfect recall in that context.
Yeah, Humans do ton of aggregation and filtering. Thus you can't remember individual days of commuting, because it is so similar, the brain just lets it go.
I'd think something like this would be added to the models eventually. Of course, over-simplifying.
I think that human memory "keeps" the next prompt by baking it into the parameters of the "neural net" (the brain) immediately
this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations.
I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.
I would say, not totally unlike a human memory. just GPT4 is limited to 8000 characters or something. Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.