[diggan]

> most of the cutting edge models that people are using day to day on local hardware fit in 12 or even 8gb of vram.

I'm not sure what your idea of "day to day" use cases are, but models that fit in 12GB of VRAM tend to be good for like autocomplete and not much more. I can't even get those models to chose the right tool at the right time, even less be moderately useful. Qwen2.5-32B seems to be the lower boundary of a useful local model, it'll at least use tools correctly. But then for "novel" (for me) coding, basically anything below O1 is more counter-productive than productive.

[wing-_-nuts]

Yes I was gonna mention that Qwen model from the deepseek folks as maybe an exception