[ajcp]

Just provide it the localhost:port for your instance of LM Studio/text-generation-webui as the Azure OpenAI endpoint in the config. Should work fine, but going to confirm now.

EDIT: Okay, this repo is a mess. They have "OpenAI" hardcoded in so many places that it literally makes this useless for working with Azure OpenAI Service OR any other openai style API. That wouldn't be terrible once you fiddled with the config IF they weren't importing the config BEFORE they set default values...

[A4ET8a8uTh0]

Yeah, I was gonna say, I ran into all sorts of issues, but I couldn't immediately tell if it is my weird config or something. I don't want to give up on it, because the idea is interesting, but I will admit I only have so much time to spend on this today.

[ajcp]

I finally got it straightened out after I unborked their hardcoding and implemented my own non-OpenAI/Azure OpenAI Service client.

[potatoman22]

Care to share the fork?

[ajcp]

Absolutely but you'll have to give me a couple hours. I initially created a custom LM Studio client, but I've gone off the deep-end and am going to implement the Ollama client.

[lylabs]

bump please share via github we'll be happy to contribute