[kamranjon]

I'm really happy this is one of the top comments here, I am fully local as well.

Just wanted to leave a note for folks who might not have the memory to run a big 32gb model - I just found out there are some pruned models that have really good performance and If I had a smaller machine I might try this pruned unsloth Q4 quant of GLM 4.7 flash that sits at 14gb: https://huggingface.co/unsloth/GLM-4.7-Flash-REAP-23B-A3B-GG...

I usually use LM Studio for this type of thing but unsloth has their own studio type app that might be even better suited for these quants.

I used GLM 4.7 flash as my main model for months and it was an incredibly tenacious model and very very fast - I think on restricted hardware, this could be a great choice.

[MrDrMcCoy]

It's a good model for sure, but it's training set is based on some pretty old stuff. Would be good to post-train any languages / frameworks you care about being current into a LoRA for it. I even had to do that for the newer Qwen3.6, but it wasn't too hard thanks to Unsloth.

[kamranjon]

Curious what data sets you used?

[MrDrMcCoy]

Made my own from the latest docs of Ansible and Netbox for ops work. Even recent models seem to be trained on versions before some very breaking changes.