[SwellJoe]

The Gemma 4 QAT 4-bit models are basically indistinguishable from the 8-bit or 16-bit versions. I tested, and many benchmarks agree. For models where there isn't a QAT version, 6-bit is where it starts to degrade, but only a little if using dynamic quantizations like unsloth provides.

When I use Qwen, I use the 8-bit quantization. But, when I use Gemma there is literally no reason to use anything other than the 4-bit QAT. It's faster, it's smaller, there's more room for context, and it performs at something like 99+% of the 16 bit.

[pbgcp2026]

I personally found in my work (long non-programming business-related context, 150k+) that only Gemma 4 31B full precision and full context works reasonably well. MoE Gemma 4, even non–quantised, makes clear logical mistakes or omissions. On real tasks. If I compare them on tiny tests – yes, sure, even i2 quants work great! :-)