[nl]

For the most part, yes.

There are specific CUDA operations which are not guaranteed to be reproducible though, as well as some CuDNN operations which are non-determanistic without performance sacrifice, and this does cause real problems.

See https://pytorch.org/docs/stable/notes/randomness.html for some reasonable docs on this.