[kamranjon]

I think most people running deepseek locally are using DS4: https://github.com/antirez/ds4

Which provides a 2-bit quant and a mixed 2-bit/4-bit quant - which range from 80gb to 97gb

Deepseek is particularly well suited for quantization and the quality of the 2bit quants antirez has provided have been validated by folks like ggernov of llama.cpp project