Does it always need an API key to work with any models, why can't you provide GPT3.5 which is free or any other models available that work without an API key.
And the only model I would recommend to use is Sonnet 3.5
I would still recommend adding OpenAI API key - for embeddings so code search and google search work
Yeah I'm not sure how this is privacy focused if it's still using API. I have local models that could run on my CPU that have pretty big context windows that would be pretty cool if this could just plug into and keep everything local.