[btrettel]

Interesting. I had been thinking recently about grep-friendly structured text file formats given the constraints of regex. But I hadn't considered that you could design a structured text file format to be LLM-friendly given token constraints.

[tsazan]

You're right.If a format is easy to grep, it is almost always cheap to tokenize. We treat token density as a primary design constraint.