|
|
|
|
|
|
by mseri
16 days ago
|
|
|
Ok, done: https://github.com/mseri/zunzuncito
My main focus is systems with very low ram. On my M1 with 8Gb or RAM llama.cpp cannot run gemma4 26b, but this implementation works fine with 5-6 tok/s. I also replaced my crap implementation of the oai server following yours, I hope it's all right. I did add a mention to your project and this fact in the README. It is a lot more barebones than what you have, but I have to admit that it works really nicely for me |
|
|