| We ultimately switched to this approach with great success for our own product. We stumbled on its success a little by chance. We have a visual designer and for LLMs to interact with it we originally built a tool per manipulation operation type (e.g. add, edit, delete item - for various item types). We actually already had a JavaScript API with corresponding .d.ts file for scripting inside our product and one of our clients asked that we also expose this as an MCP tool. I figured sure, should be quite quick and easy, and with Claude Code's help I managed to do it in a single afternoon. We then found that the LLMs way preferred reaching for this tool, managing with it to get their tasks done with fewer mistakes along the way and in much smaller time frames. After seeing this and doing some more validation (I've also read that Cloudflare article), we ditched the other tools completely and made a cheat sheet for the LLMs on how to use our API. Because our API returns decent error messages including stack traces, even if the script fails the LLMs have no trouble making another script to fix their mistake and carrying on from where the error occurred. In hindsight it really was hardly surprising as LLMs are already aiming to be as good as possible at coding and with JS being so popular I imagine it's particularly good at it. |
we did some iterations specially on the typing and error messages, as you mentioned, that help a lot in the reliability