[martinald]

FWIW for the first time in 2+ years I managed to compile llama.cpp with ROCm out of the box and run a model with no problems* on Linux (actually under WSL2 as well), with no weirdness or errors.

Every time I have tried this previously it has failed with some cryptic errors.

So from this very small test it has got way better recently.

*Did have problems enabling the WMMA extensions though. So not perfect yet.

[halJordan]

If this has been an issue for two years, then it's not rocm or llama.cpp problem.

[martinald]

Oh I'm sure you are right its operator error, but I'd always have some issue installing rocm and getting the paths right or something. This is the first time I've managed to install rocm following the commands exactly and then compile llama.cpp without having to adjust anything.

BTW, this kind of dev experience does really matter. I'm sure it was possible to get working previously; but I didn't have the level of interest to make it work - even if it was somewhat trivial. Being able to compile out of the box makes a big difference. And AFIAK this new version is the first to properly support WSL2, which means I don't have to dual boot to even try and get it working. It's a big improvement.

[vlovich123]

You can blame the user for not using the tools correctly or the manufacturer for making difficult to use tools that aren’t straightforward or don’t work in various non happy path conditions (ie unreliable installers).

For example, to this day installing MSVC doesn’t make a default sane compiler available in a terminal - you have to open their shortcut that sets up environment variables and you have to just know this is how MSVC works. Is this a user problem or Microsoft failing to follow same conventions ever other toolchain installer follows?