[impresburger]

Agree. We've designed a mechanism to enable any of the agents to ask for input from the user, but we haven't implemented it yet. Especially for more complex use cases, or use cases where the datasets are large and training runs are long, being able to interrupt (or guide) the agents' work would really help avoid "wasted" one-shot runs.

Regarding more complicated models and demanding users, I think we'd need:

1. More visibility into the training runs; log more metrics to MLFlow, visualise the state of the multi-agent system so the user knows "who is doing what", etc. 2. Give the user more control over the process, both before the building starts and during. Let the user override decisions made by the agents. This will require the mechanism I mentioned for letting both the user and the agents send each other messages during the build process. 3. Run model experiments in parallel. Currently the whole thing is "single thread", but with better parallelism (and potentially launching the training jobs on a separate Ray cluster, which we've started working on) you could throw more compute at the problem.

I'm sure there are many more things that would help here, but these are the first that come to mind off the top of my head.

What are your thoughts? Anything in particular that you think a demanding user would want/need?

[Stiopa]

Those sound like great next steps! Even without parallel runs, just by having the ability to tweak things during model builds would be super valuable; sort of like how many of the AI IDEs today (Cursor) make changes in increments and, crucially, always ask you whether to proceed or not (and how). At least that's what comes first to mind!

[vaibhavdubey97]

Absolutely! We started off thinking that we wanted to automate the whole way in one go and then add restrictions and interruptions based on areas where users face issues. This is great feedback so thank you!