[btucker]

I hacked this together last fall to let you use Apple Foundation Models with llm: https://github.com/btucker/llm-apple . To enable that I built python bindings with Claude Code: https://github.com/btucker/apple-foundation-models-py

Unfortunately, I found the small context window makes the utility pretty limited.

[troyvit]

Yeah I think you hit on the head a good way to use it though. I'm not on MacOS but KDE has a little tool called krunner[1] that lets you perform simple tasks from a small pop-up on your desktop. It would be cool if I could do slightly agentic things from there with a local model like ask what the capital of Austria is, or what's the current exchange rate between two currencies.

Then save the heavy lifting for the big boys.

[1] https://userbase.kde.org/Plasma/Krunner

[jasomill]

I've used Alfred[1] for many years on macOS, and more recently the PowerToys Command Palette[2] on Windows.

Both are easily extensible, so you could certainly wire either up to the local (or hosted) LLM of your choice.

Side note: I just noticed the extension example in the Command Palette demo reel searches HN.

[1] https://www.alfredapp.com/

[2] https://learn.microsoft.com/en-us/windows/powertoys/command-...