[dvt]

Starting work on a product where I'll need RAG + some language model (maybe llama) and Kalosm seems interesting. However, I'd like to package the model with the app. I don't really like the new trend of on-demand downloading the model via a library in some random cache folder on the user's computer (which services like Huggingface have popularized).

Is there any non-hacky way of doing this?

[Evan-Almloff]

Yes, you can set the source to any local file instead of a huggingface model. Here is a example: https://gist.github.com/ealmloff/3398d172180fa783f043b4a2696...