[whimsicalism]

Eh. I love pytorch, but it can definitely be difficult to reason about at times. For instance, due to async dispatch on GPU, you could get assertion errors where a line fails, but the real error was actually several lines above.

That was difficult to reason about.

[atorodius]

Wouldnt this be fixed by CUDA_LAUNCH_BLOCKING=1? Or putting a bunch of torch.cuda.synchronizes in the suspected lines.

[whimsicalism]

lol whoops yeah that would definitely solve the problem. I wasn't familiar with `CUDA_LAUNCH_BLOCKING` but `torch.cuda.synchronizes` does work.