Hacker News new | ask | show | jobs
by jacooper 875 days ago
Supports* dedicated*
1 comments

While not being officially supported, rocm runs just fine on my 6700XT, i just have to set an env var(export HSA_OVERRIDE_GFX_VERSION=10.3.0)
Really? Does everything run? Even AI stuff? Do you have any links where I can read more about that?
Everything I've tried to get running, worked quite smoothly. Although I only tried LLMs via llama-cpp and stable diffusion via ComfyUI. I don't see any reason why other AI stuff wouldn't work as long as it supports rocm.

Also I only tried it on linux, AFAIK windows is a lot more difficult to get running, if it works at all...

With llama-cpp, I successfully tried various LLMs(e.g. LLAMA 13B, Mixtral etc) with very solid performance. Even for models that don't fit in VRAM completely, performance can be surprisingly solid, as long as you compile with AVX extensions. (and your CPU supports those)

Stable Diffusion via ComfyUI also works very well. However, be aware of VRAM limitations with the larger SDXL variants, especially when running a heavy desktop environment.

Regarding setup guides/links, there isn't a good centralized resource sadly, so some tinkering is needed. Unlike some of those CUDA 1-click solutions, ROCm requires more manual setup, especially for the models only unofficially supported.

Here are a couple of links that might be helpful:

https://old.reddit.com/r/LocalLLaMA/comments/18ourt4/my_setu...

https://old.reddit.com/r/StableDiffusion/comments/ww436j/how...

https://rentry.org/eq3hg

In general the r/localllama & r/StableDiffusion subreddits are good places to search for info.