[smeeth]

You're conflating two concepts: native bit width and post-training quantization.

Consider two models: one is 16B and trained natively in 2 bits; one is 8B and trained natively in FP4. These models have the same total number of weight bits, but one has twice as many parameters. There is no real reason the FP4 one should be better just because it's FP4. It might be, but that is an empirical question, not a general rule.

Post-training quantization is another thing entirely. Taking a model trained at higher precision and forcing it down to 2 bits is going to hurt performance, often very badly. But this was never my point.