Hacker News new | ask | show | jobs
by diggan 536 days ago
> Graphiti uses OpenAI for LLM inference and embedding. Ensure that an OPENAI_API_KEY is set in your environment. Support for Anthropic and Groq LLM inferences is available, too.

Don't have time to scan the source code myself, but are you using the OpenAI python library, so the server URL can easily be changed? Didn't see it exposed by your library, so hoping it can at least be overridden with a env var, so we could use local LLMs instead.
1 comments
diggan 536 days ago
On second look, it seems like you've already rejected a PR trying to add local LLM support: https://github.com/getzep/graphiti/pull/184

> We recommend that you put this on a local fork as we really want the service to be as lightweight and simple as possible as we see this asa good entry point into new developers.

Sadly, it seems like you're recommending forking the library instead of allowing people to use local LLMs. You were smart enough to lock the PR from any further conversation at least :)

link
roseway4 536 days ago
You can override the default OpenAI url using an environment variable (iirc, OPENAI_API_BASE). Any LLM provider / inference server offering an OpenAI-compatible API will work.
link
diggan 536 days ago
Granted they use the `openai` python library (or other library/implementation that uses that same env var), hence my question in the previous-previous comment...
link