[lumost]

This gets a bit tricky. Over very long task contexts (1M tokens) or with prompt compression (10s of millions of tokens) the model can alter its priors based on updated evidence. This form of knowledge based learning is not necessarily robust, but demonstrably does occur.

[MarkusQ]

"the model can alter its priors"

The model doesn't have high-level priors in the Bayesian sense (though you could have priors about it).

The low-level priors it does have (the weights) are not modified by the context.