[michaelt]

> Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM considers likely.

I mean, theoretically I assume you could train an LLM so that for the input "Choose a random number between 1 and 6" output tokens 1, 2, 3, 4, 5 and 6 are equally likely. Then the sampling process would produce a random number.

Of course, whether you could teach the model to generalise that more broadly is a different matter.