Hacker News new | ask | show | jobs
by littlestymaar 640 days ago
> To use aiq label, you'll also need an OpenAI key. The other commands can be used on their own with no API key, as they run locally on your computer. Set the key as an environment variable

There are plenty of providers and self-hosted solutions that expose an OpenAI-compatible API so instead of just asking for on OpenAI key you could ask for an API end-point URL (optional with OpenAI as the default if you want to keep the current UX) and a authentication key (optional as well since self-hosted stuff don't need it).
2 comments
coolspot 640 days ago
They provide that with base URL parameter:

--api_base_url: You can use this flag (or the OPENAI_BASE_URL environment variable) to pass a different API URL from the default "https://api.openai.com/v1". This allows you to use other OpenAI-compatible LLM providers like TogetherAI, self-hosted vLLM or Ollama, etc.

link
andersonbcdefg 639 days ago
Hello! Yes, the other person who replied to you beat me to it, but this already works with any LLM provider that is OpenAI-compatible, via providing a custom API base URL. For hosted providers, this includes Together AI, Mistral, and I think even Google has an OpenAI-compatible endpoint for Gemini now. This should also work with self-hosted options like vLLM and Ollama. I led with OpenAI in the quickstart as this is the provider that most people are likely familiar with, and easiest to get started.
link