[razodactyl]

From my experience there's definitely context beyond the current set of LLM state. It's how they're able to regurgitate facts or speak at all.

[mlyle]

> regurgitate facts or speak at all.

Most of that is encoded into weights during training, though external function call interfaces and RAG are broadening this.