[mseri]

Nice!

I had done the exact same with gemma4 26b, both for my Intel laptop and for my M1 with 8Gb RAM (with also q4 and turboquant). I don’t use it much since there are dumber but way faster models to run, but I should clean up the code and make it available

[mseri]

Ok, done: https://github.com/mseri/zunzuncito My main focus is systems with very low ram. On my M1 with 8Gb or RAM llama.cpp cannot run gemma4 26b, but this implementation works fine with 5-6 tok/s.

I also replaced my crap implementation of the oai server following yours, I hope it's all right. I did add a mention to your project and this fact in the README. It is a lot more barebones than what you have, but I have to admit that it works really nicely for me

[dwa3592]

glad to know. thanks :)