[theshrike79]

The point IMO in "agent-native CLIs" is to make them match the statistical average.

Let the Agent use the CLI and if it guesses the wrong option, you make that the RIGHT option.

Every time it doesn't guess something right, you change it.

[pmontra]

I would naively suppose that the agent is able to read the man page or run the help command of the tool. They usually contain plenty of information. But bending the tool to suit the agent has some value. The GNU-AI suite of userland tools? Unfortunately it's possible that every model will settle on a different average. If that's the case we can't bend to every model. Models will have to bend to whatever we want to use.

[theshrike79]

Of course it can read the man page and run cmd --help.

Now you've wasted context on, what? Learning how to use the tool. And it will waste context on it every single time. (You can write skills to mitigate this a bit, but still).

The alternative is to make the tool work as the user (an LLM in this case) expects it to work, without having to resort to the manual.

[riknos314]

If the parameter names mostly standardize across tools because the models learn to predict those names, then humans will also learn to predict those flag names so this actually has the potential to make tools more human friendly and easier to learn.

[tfrancisl]

> Let the Agent use the CLI and if it guesses the wrong option, you make that the RIGHT option

This sounds backwards and presumes that the statistics machines which are LLMs are getting it right when they "average" out to the wrong command. No, fix the agents behavior, dont change the CLI to accommodate it.

[alchemist1e9]

I don’t remember exactly the specific examples off the top of my head (some are definitely ffmpeg commands) but I do know that when LLMs keep hallucinating command line flags that don’t exist for that specific command their “suggestion” is actually very reasonable and so many developers are adding support to their tools for common hallucinations.

[tfrancisl]

Not to belabor my point, but I think "adding support to tools for common hallucinations" is a bad idea. Sounds like something a vibecoded project being spammed with issues by agents might do. Not so much a serious, mature project, though.

[alchemist1e9]

Well we will have to agree to disagree because my understanding of what has been generally the case is that the LLMs might vibe-coding spam, that’s true, but the interesting difference is generally speaking their “suggestions” are very reasonable and represent in hindsight useful changes that make the commands more useful for everyone, humans included.

[QuercusMax]

If an option exists but it's got a poorly named flag, adding a flag alias is probably a good idea for usability in general. Most CLI tools probably don't report telemetry about failed executions, though, cuz that would be very creepy.

[rsalus]

the real solution is to simply provide hints in responses so that the model may self-correct, e.g., recommended next actions, describe commands to get schema definitions, etc.