[cududa]

I mean it’d take minutes of research to realize people are successfully and efficiently running 4-bit quantized GLM 5.2 on MacStudio 512GB M3 Ultras at over 60 tok/s. K2 2.7 is quite literally designed for 4 bit quantization and runs even better.

This is already a thing