[tacticalglare]

What would this need to be compatible with Ollama? I would love to run this with a local LLM!

[shriyansha]

We tried Gpt-4o, Anthropic haiku, Groq's 8b tool preview, Llama 3.2 8b tool support via Groq and all worked well.

With Ollama in our machines, we didn't get good results maybe because of model limitation on function calling.

We are experimenting with more models and smaller models so that running local with local models becomes possible.

Although if you check the readme and example config file there is a way to give it a try with ollama.

[Smilinrobin]

You can already try other LLMs. Steps in docs.