[samuel]

That's not an inherent property of the system. You can choose the most likely token(topk=1) and it will be deterministic (at least in theory, in some hardware setups it might be trickier)