[moritzwarhier]

> However, I believe it also shows that "general-purpose" models are not really general purpose and can't really step out of their assistant role. A common and often promoted prompting technique is to prompt a model to "behave like ..." or "you are a ..." which means these instructions do not really work.

I think you're still confusing model and "LLM app" there.

I'm not that versed myself in these things, but you can, for example, look at the conversation templates, stop markers etc. in open weight models on HuggingFace, or play around with these things by yourself and modify them using llama.cpp or ollama (the things I mention in this paragraph are, AFAIK, not part of the model). These, and parameters like temperature, sampling etc. are just the things that can be controlled without touching the model.

Of course, frontier models and their uses have supposably a lot more machinery built around them to orchestrate their usage, apart from even chatbots defaulting to "agentic" behavior for many use cases.

And models still are specialized, and fine-tuned for instruction usage, so things like the conversation template, system prompts won't be enough to bend the characteristics of such a model in all desired directions. But "general-purpose model" has become a very fuzzy term by now.

[orbital-decay]

The model is trained to follow the default template pretty closely, breaking it usually results in much worse performance and better output variance. Certain models with synthetic data in pre-training can melt down completely. At some point into this breakage you can just take the base model and it will be better.

If you want to use a custom chat scheme, use it as an overlay, don't break the default chat/tool use/reasoning template.

[moritzwarhier]

Thanks!

Matches my superficial experiments with trying to tweak Ollama's "modelfile" using some LLaMa- or gpt-oss-based instruction-tuned model as "base".

I need to experiment more with base models. The time from the end of 2019 onwards, when I first came across talktotransformer, it felt so magical.

Getting meaningful things out of these things can feel so... restraining.

And on the other hand: I'm tbh freshly stuck in the stage of being amazed at what current frontier coding models and apps can do.