[rayboy1995]

Thanks!! I had disabled that previously while debugging, I can confirm this is helping accuracy from what I can tell so far. (And speed since the cache is preserved more often!)

[satvikpendem]

Use the MTP models which 2x token generation speed, for example: https://unsloth.ai/docs/models/qwen3.6#mtp-guide

[rayboy1995]

Very interesting I'll have to check this out thank you. This is why I love HN.