[freehorse]

I think that a big issue here is the chat UX implementations in most IDEs. In contrast, in zed, in the text-based chat threads, one can directly edit an LLM's answer, making actually fixing bugs oneself much less complicated in terms of UX. In a classical chat interface, where you cannot edit an LLM's answer, you have to either continue the thread, start a new thread with a different prompt asking it to avoid the issue, or fix the code and provide it as context again. If you do it in a new chat, you lose the context of the previous chat (you need to provided it again). If you continue the chat with the added code, you have dirty context (which makes llms perform worse). Moreover, they most probably require you to get in and out of the chat to do things, which is not ideal UX compared to just prompting again. The fact that I can just change the generated code in the chat itself, run it in the repl, and continue from there is the best UX imo for when such problems occur. I would like if more IDEs actually accepted something similar.

Of course, my workflow is very repl based and I assume different experiences exist for other contexts. Moreover, agent workflows have probably different pros and cons. It is great timewise that they can test code and fix bugs themselves, but essentially that's a bit like automating this prompt-reprompt loop. It saves time compared to that (at the cost of a lot more $$$ in tokens overall), but if the agent fails to solve the issues, yet again we are at square zero. What helps a lot is that work is done inside the codebase at least, which is supposed to be a good thing, but personally I have not understood how to use agents efficiently tbh.