[big-chungus4]

A better solution might be not to constrain the generation, but to remove invalid fields from the tool call in the assistant message. So on the next turn, the model receives chat history which contains it's tool call, but without extra arguments. You can do that in OpenAI chat completions / responses, not sure about Anthopic API.

There is still a downside, sometimes the model really wants to include an additional argument for whatever reason it reasoned towards, and it needs the error message to understand that the argument doesn't exist. Otherwise if the argument is manually removed from it's tool call, the model will think that it accidentally left out the argument and start retrying and might go into a loop.

[the_mitsuhiko]

That kills the KV cache unfortunately. In some models (Gemini) I also doesn’t work because there is a signature on the model messages.