[johndough]

It would be great if this could be combined with quantization-aware finetuning. In my experience, Qwen3.6-27B has much fewer repetitions at Q6 quantization level as compared to e.g. Q4, but that leaves little space for context on my 24GB RTX 3090.

[storus]

They are orthogonal; preference optimization like RLHF can be done on the base model which can later be quantized, or it could be done on a new LoRA that is then converted to QLoRA.