Hacker News new | ask | show | jobs
by liuliu 14 days ago
Note that 3.5 9B cannot do thinking (while 3.6 27B can, pretty effectively, quite verbosely).
1 comments

3.5 9B can do thinking. Its just disabled by default in its gguf chat template.
It is disabled because it doesn't work :) Try it and see the doom loop it gets itself in.
I had no failure mode in 3.6 9B thinking with llama.cpp. After release, there were updates for both model and llama.cpp.