[novaRom]

Gemma 4 12B QAT is amazing - agents run very fast, and it's really very smart, at least in my agent's harness domain which is GNU software development - on par with frontiers like GPT Sol, DeepSeek, or Claude - Why to buy those expensive tokens if a local tiny model performs so well?

[gwerbin]

What's your harness setup? I haven't had this kind of on-par success with any local LLM yet.

[sosodev]

They’re exaggerating or have a very simple way of using these models. The Gemma 4 series, even at 31B, is nowhere near the frontier. They’re great models, but you will notice a huge difference for complex tasks.

The best local agentic coding experience I’ve had so far is Qwen3.6-27B with Pi.

[xienze]

> They’re great models, but you will notice a huge difference for complex tasks.

Yes and no. I think where frontier models really blow small models away is in how thorough they are in order to infer your intentions and how best to accomplish them. So you can tell Claude "change this code to make it do X", whereas a Qwen3.6-27B or Gemma4-31B can do the same, but you have to be a lot more thorough, i.e., "change this code to make it do X, but first, let me explain the concept of X as I see it and some notes about things to avoid or pay attention to while you're doing it." So for best success with small models you really need a big toolbelt of skills and MCPs.

[verdverm]

same with opencode, next on my evals.list are the fine tunes from big model traces, and then my own if ftuning looks reasonable cost/time wise

I have two main tasks I want to see if I can improve, coding and doc understanding/summary