|
|
|
|
|
|
by orbital-decay
8 days ago
|
|
|
The model is trained to follow the default template pretty closely, breaking it usually results in much worse performance and better output variance. Certain models with synthetic data in pre-training can melt down completely. At some point into this breakage you can just take the base model and it will be better. If you want to use a custom chat scheme, use it as an overlay, don't break the default chat/tool use/reasoning template. |
|
|
Matches my superficial experiments with trying to tweak Ollama's "modelfile" using some LLaMa- or gpt-oss-based instruction-tuned model as "base".
I need to experiment more with base models. The time from the end of 2019 onwards, when I first came across talktotransformer, it felt so magical.
Getting meaningful things out of these things can feel so... restraining.
And on the other hand: I'm tbh freshly stuck in the stage of being amazed at what current frontier coding models and apps can do.