[Sharlin]

It should have a zero chance of generating different output if the temperature is set to zero as in TFA. LLMs are not stochastic algorithms unless you add entropy yourself. Of course most people just use ChatGPT with its default settings and know nothing about the specifics.

The point is, though – somehow the model has memorized these passages, in a way that allows reliable reproduction. No doubt in a super amorphous and diffuse way, as minute adjustments to the nth sigbits of myriads of floating-point numbers, but it cannot be denied that it absolutely has encoded the strings in some manner. Or otherwise you have to accept that humans can't memorize things either. Indeed given how much our memory works by association, and how it's considerably more difficult to recount some memorized sequence from an arbitrary starting point, it's easy to argue that in some relevant way human brains are next-token predictors too.

[suprjami]

The model has taken the input passages from its training data and tokenised it into weights. Don't humanise it by saying it has "remembered" anything. It does not and cannot remember sequences.

Yes, if you reduce temperature to zero and set the same random seed, you should get the same output tokens for a given set of input tokens.

However, there is no guarantee the output for a given seed will be the correct expected output.

For example, there logically must be a model and seed where providing the lord's prayer as input for completion produces a Metallica song as output, because that's a viable set of input tokens: https://genius.com/Metallica-enter-sandman-lyrics

That seed is no more or less valid than any other seed which completes the actual lord's prayer or which provides something completely different. All those seeds are just predicting their next token.

If people want that sort of exact reliable retrieval of sequences, and for the sequences to be "correct", then an LLM is the wrong tool for the job.