Hacker News new | ask | show | jobs
by api 364 days ago
I wish LM Studio had a pure daemon mode. It's better than ollama in a lot of ways but I'd rather be able to use BoltAI as the UI, as well as use it from Zed and VSCode and aider.

What I like about ollama is that it provides a self-hosted AI provider that can be used by a variety of things. LM Studio has that too, but you have to have the whole big chonky Electron UI running. Its UI is powerful but a lot less nice than e.g. BoltAI for casual use.
2 comments
rhet0rica 363 days ago
Oh, that horrible Electron UI. Under Windows it pegs a core on my CPU at all times!

If you're just working as a single user via the OpenAI protocol, you might want to consider koboldcpp. It bundles a GUI launcher, then starts in text-only mode. You can also tell it to just run a saved configuration, bypassing the GUI; I've successfully run it as a system service on Windows using nssm.

https://github.com/LostRuins/koboldcpp/releases

Though there are a lot of roleplay-centric gimmicks in its feature set, its context-shifting feature is singular. It caches the intermediate state used by your last query, extending it to build the next one. As a result you save on generation time with large contexts, and also any conversation that has been pushed out of the context window still indirectly influences the current exchange.

link
diggan 363 days ago
> Oh, that horrible Electron UI. Under Windows it pegs a core on my CPU at all times!

Worse I'd say, considering what people use LM Studio for, is the VRAM it occupies up even when the UI and everything is idle. Somehow, it's using 500MB VRAM while doing nothing, while Firefox with ~60 active tabs is using 480MB. gnome-shell itself also sits around 450MB and is responsible for quite a bit more than LM Studio.

Still, LM Studio is probably the best all-in-one GUI around for local LLM usage, unless you go terminal usage.

link
SparkyMcUnicorn 364 days ago
There's a "headless" checkbox in settings->developer
link
diggan 363 days ago
Still, you need to install and run the AppImage at least once to enable the "lms" cli which can later be used. Would be nice with a completely GUI-less installation/use method too.
link
t1amat 363 days ago
The UI is the product. If you just want the engine, use mlx-omni-server (for MLX) or llama-swap (for GGUF) and huggingface-cli (for model downloads).
link
diggan 363 days ago
Those don't offer the same features as LM Studio itself does, even when you don't consider the UI. If there was a "LM Engine" CLI I could install, then yeah, but there isn't, hence the need to run the UI once to get "the engine".
link
SparkyMcUnicorn 361 days ago
I haven't used this and maybe it doesn't solve the problem you're describing, but might be worth looking at.

https://github.com/lmstudio-ai/lms

https://lmstudio.ai/docs/cli

link