Hacker News new | ask | show | jobs
by dur-randir 45 days ago
How do I connect it to a local llama.cpp instance?
1 comments

It supports LMStudio or you can start a local endpoint, then run

OPENAI_COMPATIBLE_CUSTOM_KEY="xxx" dirac -y --provider "https://localhost/v1" --model <model_name> "hi..."