The difference is a yolo’d script needs to be verified everytime it’s generated. So it makes sense to generate it and verify it once (a cli command or mcp call).
I think OP’s example is bad. As that (if I understand the example correctly as I don’t really know what liveclip is) sounds like something worth generating a script for once and reusing it. However, if you’re looking to book a flight, you would prefer an airline cli or mcp server or a nodejs script using puppeteer to automate the browser or even some airline sdk to buy a ticket?
What doesn’t make sense is generating a script to achieve a task without clear feedback loop for verifying the script correctness. In OPs example, there is no way for the LLM to very the correct final result since it’s unknown. It can “code review” the code looking for bugs, but bugs in scripts happen.
Yeah, of course. The MCP server tools do use Python in the backend but just look at the affordances it provided in the args -- named columns, a fixed match/replace API, a way to specify the range of rows. The LLM is not gonna create a little library with these affordances in an ephemeral REPL it's just gonna YOLO some pandas code that you hope did what you wanted and addressed the right columns, had the right types, etc
I feel like I am having a fever dream. How is an LLM that is trained on endless examples of Python, a well defined language as well, going to generate worst code than some random JSON interface?
Have you ever tried analyzing a CSV in an LLM? Maybe then you'd get what I'm saying
Heck for a lot of questions the LLM won't even write Pandas and just try to eyeball the arithmetic and be like "Yeah March 3 to March 7 values add up to..."
And often -- without using code _openly_ or in thinking -- it is wrong because it is just a "guesstimate" at best due to the nature of how LLMs generate tokens.
I think OP’s example is bad. As that (if I understand the example correctly as I don’t really know what liveclip is) sounds like something worth generating a script for once and reusing it. However, if you’re looking to book a flight, you would prefer an airline cli or mcp server or a nodejs script using puppeteer to automate the browser or even some airline sdk to buy a ticket?
What doesn’t make sense is generating a script to achieve a task without clear feedback loop for verifying the script correctness. In OPs example, there is no way for the LLM to very the correct final result since it’s unknown. It can “code review” the code looking for bugs, but bugs in scripts happen.