Hacker News new | ask | show | jobs
by gunalx 18 days ago
3.5 9B can do thinking. Its just disabled by default in its gguf chat template.
1 comments

It is disabled because it doesn't work :) Try it and see the doom loop it gets itself in.
I had no failure mode in 3.6 9B thinking with llama.cpp. After release, there were updates for both model and llama.cpp.