[paytonjjones]

I wonder if we'll start to see that pattern with every new release. Tool use likely changes rapidly, so the newest, rather than most intelligent, model may always have an edge.

[ai_fry_ur_brain]

What you mean.. The tools are all just invoking bash and terminal/cli cmds and http requests. Paradigms that have existed and stayed mostly unchanged for decades.

[paytonjjones]

These do make up a huge % of tool calls, but I don't think these make up a huge % of tool call failures.

I see models fail on tool calls that involve API requests to a specific API, internal or cloned Makefile calls, npm run commands, etc.