|
|
|
|
|
|
by brucethemoose2
1061 days ago
|
|
|
Nah, I tried Ubuntu too. The OpenBLAS package was missing on ARM, along with some other dependencies I needed for compilation. At the end of the day, even with many tweaks and custom compilation flags, the instance was averaging below 1 token/sec as a Kobold Horde host, which is below the threshold to even be allowed as a llm host. |
|
|