[koakuma-chan]

The prompt may be the same but the seed is different every time.

[biophysboy]

Could you not cache the top k outputs given a provided input token set? I thought the randomness was applied at the end by sampling the output distribution.