[fumber]

I was so excited that it is open source until i realised the model required 1.5To of VRAM. Unsloth has compressed it in 1bit at about 570gb VRAM with 75% accuracy, that's almost mac studio territory...

[nateb2022]

You can run DeepSeek v4 Flash at 4bit in ~150 GB VRAM, and it would absolutely destroy a 1 bit quant of K3.

[AmazingTurtle]

75% quality... it gets worse the longer the context.