[zmccormick7]

Those are just the defaults, and spRAG is designed to be flexible in terms of the models you can use with it. For AutoContext (which is just a summarization task) Haiku offers a great balance of price and performance. Llama 3-8B would also be a great choice there, especially if you want something you can run locally. For reranking, the Cohere v3 reranker is by far the best performer on the market right now. And for embeddings, it's really a toss-up between OpenAI, Cohere, and Voyage.

[Cheer2171]

I bet you'll get a lot more adoption if you put info about using it with local self-hosted LLMs there. I'll never trust a cloud service with the documents I want to RAG.

[zmccormick7]

Agreed. I've gotten a lot of feedback along those lines today, so that's my top priority now.