[stephenlf]

I don’t know how OpenAI prices JSON keys. They likely price it like any other token.

The model described in your paper still uses some amount of inference to generate JSON keys. Plus, each JSON key becomes part of the expanding context window. These keys aren’t free to generate.