|
|
|
|
|
|
by walrus01
2 days ago
|
|
|
I assume you mean putting only the 32B active parameters on the GPU, and the rest on a bunch of regular server DRAM like on a 768GB to 1024GB RAM server? Because Kimi K2.6 in Q4 is about 584GB GGUF size on disk and will use slightly more than that in RAM, Q8 is 595GB. https://huggingface.co/unsloth/Kimi-K2.6-GGUF |
|
|