[cschmid]

Can I also interpret this as: 'AMD's pytorch support is so abysmal that inference is 10x slower than it should be'?

[croes]

Should it not say PyTorch's AMD support?

[dannyw]

It takes two to tango. AMD is always welcome to contribute patches.

You also have to keep in mind some latest gen AMD GPUs don’t even officially support ROCm on Linux. That’s absurd.

AMD has a choice to invest more staff into ML support, they’re choosing not to.

[lhl]

I've harped on this on the past, the "official" hardware support list is tiny: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h...

But, it's worth noting there's different levels of "support." With ROCm 5.6, the 7900XT and 7900XTX RDNA3 cards, while not "officially" supported are gfx1100, which have rocBLAS/MIOpen kernels and work w/o jiggerpokery w/ PyTorch nightly and various HIPified inferencers I've tried like ExLlama.

[m00x]

From Geohotz's investigation in the matter, it doesn't look like it's a manpower issue, it's a quality/culture issue. AMD's firmware GPU division isn't amazing.