[Archit3ch]

Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.

[xtracto]

I will be delighted to try. I have a 128gb macbookpro m4 waiting for this.

[ukd1]

ds4 is pretty optimized for this btw - https://github.com/antirez/ds4

[sebastianconcpt]

This is the way.

[fragmede]

Followed you on GitHub to get notified when you are!

[arm32]

Link it already!