|
|
|
|
|
|
by spmurrayzzz
1060 days ago
|
|
|
Many of the open source inference solutions you'd use for llama ship with an openai-compatible interface (e.g. oobabooga/text-generation-web-ui, huggingface/text-generation-inference, etc). Probably can just change the API host and it would work. |
|
|