Hacker News new | ask | show | jobs
by _flux 694 days ago
> If you wanted to embed an LLM, you wouldn’t rely on a network endpoint and you could just use a llama.cpp binding.

I do want to rely on a network endpoint, as not all my computers are capable of running LLMs locally in an efficient manner.
1 comments
wokwokwok 694 days ago
> If you wanted to embed an LLM…
link
_flux 694 days ago
I read it as wanting to embed LLM functionality, but my reading might have been incorrect—and by the client library reference I understood it was referring to generic HTTP clients, but that reading might have also been incorrect :).

Per the examples this looks like an LLM-over-network library and there doesn't seem to be too many of those available for Rust.

link