[icedchai]

The "ROCm" situation with Strix Halo was pretty bad for a while. I think it finally stabilized late last year. You needed the right combo of ROCm, Linux kernel, and kernel firmware for it to work reliably.

Whenever I rebuild llama.cpp, I wind up using the Vulkan build anyway.

[anakaine]

And thats a pretty big annoyance. You need to perfectly line up all the holes in the swiss cheese for the AMD stack to work, then their dev team kicks you in the nuts anyway.

They have made a few attempts at investing in the hardware, but the software side is letting them down hard, and that part is almost entirely their own fault. They have underinvestment in their own stack, and into popular standard and Community libraries that would make it easy to use their gear.

[icedchai]

Yep, it definitely wasn't an "out-of-box" experience. This was on Ubuntu 25.10. Maybe other distros are better?

Also, when you do Python work, you have to remember to install dependencies like pytorch from an alternate repo, otherwise you wind up with the CUDA versions that only use the CPU and not the GPU through ROCm.

[lhl]

I started a project w/ the assumption that fully tuned for HIP kernels could get closer to roofline and beat llama.cpp's performance (and vs their ROCm engine it does!) but after thousands of iterations/experiments my suspicion is that RADV/ACO is actually just much better than LLVM/HIP when it comes to a lower-level scheduling etc for RDNA3.

I haven't finished comprehensive tests but I found:

- Vulkan is up to 2.25X faster for most coalesced, strided and interleave variants for memory-side scheduling/access shapes

- 3.3X faster on specific dot-path sweeps, including for scalar-dequant

- For matched LDS, Vulkan can be 8-14X+ faster (!!!) than matched HIP LDS

HIP doesn't always win against RADV/ACO, but on dispatch/runtime, it does appear to be quite a bit faster than HIP/LLVM on gfx1151 (Strix Halo). I'll be publishing sharing full data once I also run vs gfx1100...