Hacker News new | ask | show | jobs
by luckystarr 14 days ago
Tried it on Android and got "!!!!!!!!!!!!!" for answers.
2 comments
gunalx 14 days ago
The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.
link
aesthesia 14 days ago
Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior.
link
amelius 14 days ago
Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?

Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error.

link
jldugger 14 days ago
After the third !, the probability of a fourth probably skyrockets =)
link
verdverm 14 days ago
That's what happens when you quant too hard. I'm working on quant strats and evals for the same underlying qwen 27b models.

When I saw 27b on a phone, I thought not fitting, big phone, or aggressive quant. NVFP4 still takes 27G before KV cache.

link