[vergessenmir]

When working with GGUF what chat templates do you use? Pretty much every gguf I've imported into ollama has given me garbage response. Converting the tokenizer json has yielded mixed results.

For example how do you handle the phi-4 models gguf chat template?

[simonw]

I use whatever what template is baked into the GGUF file.

You can click on the little info icon on Hugging Face to see that directly.

For https://huggingface.co/matteogeniaccio/phi-4/tree/main?show_... that's this:

  {% for message in messages %}{% if
  (message['role'] == 'system')
  %}{{'<|im_start|>system<|im_sep|>' +
  message['content'] + '<|im_end|>'}}{%
  elif (message['role'] == 'user')
  %}{{'<|im_start|>user<|im_sep|>' +
  message['content'] +
  '<|im_end|><|im_start|>assistant<|im_sep|>'}}{%
  elif (message['role'] == 'assistant')
  %}{{message['content'] + '<|im_end|>'}}{%
  endif %}{% endfor %}