[mike_hearn]

That's very impressive. What's the best way to run these kernels natively on a Mac? I saw that there's a way to plug Claude into Apple's Foundation Models framework, and there's a CLI tool that can access models via that framework. It might be useful to have something so fast and good available via a small CLI tool for various purposes, especially when connected with a small suite of tools I have for things like file editing, showing, simple agentic purposes etc.

[thomspoon]

Either ollama or omlx, both are pretty dang performant. Omlx lets you run Claude code locally though as long as you bootstrap it with the right model

[mike_hearn]

Omlx is really nice, thanks for the recommendation!

[karussell]

Why would you need Omlx? For speed up?

[pornel]

Has extra KV cache on SSD, and lots more options to tweak. There's experimental TurboQuant and multi token prediction support.