Hacker News new | ask | show | jobs
by m3kw9 921 days ago
How would one go about making a .llamafile for this?
1 comments
simonw 921 days ago
Convert it to GGML and use a zip tool to add that to a llamafile package.

https://huggingface.co/TheBloke?search_models=Zephyr doesn't have a GGML for it yet but I wouldn't be surprised to see one by the end of the day.

link
pleks 921 days ago
And it's been uploaded https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree...

Getting 20toks/s on M1 mba where as LLaVa I ground to a halt. Very impressed

link