[clementmas]

I'm considering switching my function calling requests from OpenAI's API to Mistral. Are they using similar formats? What's the easiest way to use Mistral? Is it by using Huggingface?

[ru552]

easiest is probably with ollama [0]. I think the ollama API is OpenAI compatible.

[0]https://ollama.com/

[talldayo]

Most inference servers are OpenAI-compatibile. Even the "official" llama-cpp server should work fine: https://github.com/ggerganov/llama.cpp/blob/master/examples/...

[pants2]

Ollama runs locally. What's the best option for calling the new Mixtral model on someone else's server programmatically?

[Arcuru]

Openrouter lists several options: https://openrouter.ai/models/mistralai/mixtral-8x22b