[p0w3n3d]

Listen to a video made by Karpathy about LLM, he explains why made up html tags work. It's to help the tokenizer

[dotancohen]

I recall this even being in the Anthropic documentation.

[dotancohen]

Here, found it:

  > Use XML tags to structure your prompts

  > There are no canonical “best” XML tags that Claude has been trained with in particular, although we recommend that your tag names make sense with the information they surround.

https://docs.anthropic.com/en/docs/build-with-claude/prompt-...

[justsomehnguy]

My guess would be there is enough training materiel what a mere tagging sometging is enough to have a bigger SNR.

[victor106]

Could not find it. Can you please provide a link?

[p0w3n3d]

https://youtu.be/7xTGNNLPyMI?si=eaqVjx8maPtl1STJ

He shows how the prompt is parsed etc. Very nice and eye opening. Also superstition dispelling