[hyperpape]

This is a great piece of data, but only a piece of the actual question that we need to answer, which is:

For a given input, how many tokens will be used for an answer, and how high quality will that answer be?

Measuring the tokenizer is just one input into the cost-benefit tradeoff.