Hacker News new | ask | show | jobs
by terhechte 753 days ago
You can set a custom URL to use Ollama which is OpenAI API compatible. Llama3 8b runs quite fast for me on a M1 Max. I'll be making use of this feature I think (haven't tried it yet).
1 comments
jchoelt 753 days ago
No, unfortunately it's not possible to set a custom URL... OpenAI is either hard coded or using a configuration option that isn't exposed.
link
jarpineh 753 days ago
It is exposed from Advanced tab under "URL for AI API". You do have to change the model name from the preselected and offered OpenAI GPT model names. Otherwise this AI API URL isn't used. Haven't yet tested with local OAI compatible model yet, though, but did manage to break it using bad address :)
link
tinio 753 days ago
How do you change the model name from the preselected and offered OpenAI GPT model names? I don't see the option to do so.
link
weikju 753 days ago
click the name instead of the dropdown arrow

Note that to make use of ollama, you'll have to wait for the next release or upgrade to a beta release --- while you can change the API right now it's using a newer format for the payload, that's fixed in the beta/next release.

link
klaustopher 753 days ago
Advanced settings -> "URL for AI API"
link