|
|
|
|
|
|
by klardotsh
43 days ago
|
|
|
4-5 bit quants would probably fit pretty well on your rig. Check HuggingFace for Qwen3.6-35B-A3B-MTP-GGUF [1]. They've also got a cool UI thing these days to help indicate which quants of a model will run on your hardware. Full octane isn't gonna fit on much of anything south of a 128GB machine once adding KV cache. [1]: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF |
|
|