|
|
|
|
|
|
by Chordless
621 days ago
|
|
|
I'm able to run Ollama and llama.cpp on my Ryzen 4600G APU following this guide:
https://agieverywhere.com/apuguide/AMDAPU/APU_Linux Your APU should be similar, just faster. There are some magic environment variables you want to set to get ROCM to work with this technically unsupported APU:
HSA_OVERRIDE_GFX_VERSION=9.0.0
HSA_ENABLE_SDMA=0 Performance is not great, but slightly better than running inference on the CPU, with the bonus that your CPU is essentially free for other tasks even while running LLMs. |
|
|