Show HN: Gerbil – an open source desktop app for running LLMs locally

[lone-cloud]

Gerbil is an open source app that I've been working on for the last couple of months. The development now is largely done and I'm unlikely to add anymore major features. Instead I'm focusing on any bug fixes, small QoL features and dependency upgrades.

Under the hood it runs llama.cpp (via koboldcpp) backends and allows easy integration with the popular modern frontends like Open WebUI, SillyTavern, ComfyUI, StableUI (built-in) and KoboldAI Lite (built-in).

Why did I create this? I wanted an all-in-one solution for simple text and image-gen local LLMs. I got fed up with needing to manage multiple tools for the various LLM backends and frontends. In addition, as a Linux Wayland user I needed something that would work and look great on my system.

[WillAdams]

The big feature which I would like to see is a way to easily interact with the content of the local filesystem --- I have a prompt for re-naming scans based on parsing their content which I've been using in Copilot --- recent changes require that I:

- launch Copilot

- enter a prompt to get it into Copilot Pages mode

- click a button to actually get into that mode

- paste in the prompt

- drag in 20 files

- wait for them to upload

- click the button to process the prompt on the uploaded files

- quit Copilot, launch Copilot, delete the conversation, quit, launch Copilot and have it not start, which then allows repeating from the beginning

It would be much easier if I could just paste in the prompt specifying a folder full of files for it to run on, then clear that folder out for the next day's files and repeat.

Would that be something which your front-end could do? If not, is there one which could do that now? (apparently jan.ai has something like this on their roadmap for 0.8)

[tell_me_whai]

Hey, funny finding you comment as I've actually recently been developing a CLI app to improve LLM integration in my filesystem. Not sure what you are doing with your files, but maybe it could be useful for you too! You can check it out on my github (https://github.com/gael-vanderlee/whai) and hopefully that can help you with your use case. Look into roles in particular, as they allow you to save and reuse specific workflows.

[lone-cloud]

I believe what you're describing is outside of the scope of Gerbil. Gerbil is not an LLM front-end, but Gerbil will run your LLM and seamlessly integrate (orchestrate) it with a custom front-end from the list in my original message. I believe this functionality will need to live in a custom front-end. I'm curious how jan.ai is planning on handling this. I'm guessing they're writing their own custom front-ends which is probably tightly integrated with their system.

[WillAdams]

Very hopeful of the multi-file stuff from jan.ai --- in the meanwhile, it's easier using Co-pilot for this than:

- isolating 50 files at a time

- dragging them into Adobe Acrobat

- closing each w/ ctrl w

- tapping enter to confirm saving

- typing the Invoice ID

- repeating until all 50 have been done, then remembering to quite Adobe Acrobat so as to re-launch it and repeat (can't leave it running, because there is a (reported) bug where after doing this several times, it stops saving)

- running a batch file made from a concatenated column in a spreadsheet to rename the files

The next question is when there will be an LLM front-end which can:

- open each file in a folder, parsing the content

- open each file in a PDF viewer

- fill in the entry fields of a Java application

- wait for the user to review both windows, if necessary, correct/update what was entered and save, then repeat for the next file

Ah well, job secure, even when that happens (though maybe hours would be cut back?) --- the big question is when LLMs will be reliable enough that human review is no longer viewed as worth the expense of a salary.

[tell_me_whai]

Does this allow for mixing LLMs and Image Gen? I find LLMs really useful to generate image prompts that diffusion models understand (which can be tedious to do manually). Although you need very detailed system prompts to teach what Image Gen models expect.

[lone-cloud]

That's how the pros do it. Yes, you can load both a text and image gen models at the same time. Needless to say you'll need a very beefy GPU(s) to do this so I wouldn't recommend it unless you know exactly what you're doing as generally you'll want to max out your VRAM for one model at a time for the highest quality results. Open webui and sillytavern allow both text and image gen from the same UI although I wouldn't recommend it for advanced users. Otherwise Gerbil will give you multiple pages to toggle through via the titlebar dropdown.

[radial_symmetry]

I like that it has image generation without all the complication of ComfyUI. Can it load LoRA?

[lone-cloud]

Gerbil's built-in image generation is based on "StableUI" and I also prefer its super simple UI. Yes, you can load your own LoRa from the "Image Generation" tab. Gerbil also includes the optional ComfyUI integration from the settings for very advanced users. Its graph-based UI is a bit too advanced for me personally.

[throwaway81998]

Serious question, not a "what's the point of this" shitpost... My experience with local LLMs is limited.

Just installed LM Studio on a new machine today (2025 Asus ROG Flow Z13, 96GB VRAM, running Linux). Haven't had the time to test it out yet.

Is there a reason for me to choose Gerbil instead? Or something else entirely?

[lone-cloud]

Holy, your machine is a beast. 96GB of VRAM is pretty insane. I've been running a single 16GB VRAM AMD GPU. At the bottom of Gerbil's readme I listed out my setup where I use a 27b text gen model (gemma 3) but you'll be able to use much larger models and everything will run super fast.

Now as for your question, I started out with LM studio too, but the problem is that you'll need to juggle multiple apps if you want to do text gen or image gen or if you want to use a custom front-end. As an example, my favorite text gen front-end is "open webui" which gerbil can automatically set up for you (as long as you have Python's uv pre-installed). Gerbil will allow you to run text, image and video gen, as well as set up (and keep updated) any of the front-ends that I listed in my original post. I could be wrong but I'm not sure if LM studio can legally integrate GLP licensed software in the same way that Gerbil can because it's a closed source app.

[throwaway81998]

Thanks for the reply, I'll give Gerbil a try.

[A4ET8a8uTh0_v2]

Not OP, but I am running ollama as a testing ground for various projects ( separately from gpt sub ).

<< Is there a reason for me to choose Gerbil instead? Or something else entirely?

My initial reaction is positive, because it seems to integrate everything without sacrificing being able to customize it further if need be. That said, did not test it yet, but now I will.