[yuvalr1]

This means the leading UI for LLMs - the chat - is the wrong UI, at least for some of the tasks. We should instead have a single query text field, like in search engines, that you continue to edit and refine, just like in complex search queries.

[freehorse]

I like zed's approach, where the whole discussion is a plain text file you can edit as any text, which gives you the ability to change anything in the "discussion" regardless if it was generated by you or the llm. It makes stuff like that much simpler, ie you can correct simple stuff in the llm's response without unecessary back and forths, you can just cut parts out of the discussion to reduce context size or guide the discussion where you actually want removing distractions etc. I don't understand why the dominant approach is an actual, realistic chat interface where you can only add a new response, or in best case create "threads".

[diggan]

> I don't understand why the dominant approach is an actual, realistic chat interface where you can only add a new response, or in best case create "threads".

I'm not 100% sure either, I think it might just be a first-iteration UX that is generally useful, but not specifically useful for use cases like coding.

To kind of work around this, I generally keep my prompts as .md files on disk, treat them like templates where I have variables like $SRC that gets replaced with the actual code when I "compile" them. So I write a prompt, paste it into ChatGPT, notice something is wrong, edit my template on disk then paste it into a new conversation. Iterate until it works. I ended up putting the CLI I use for this here, in case others wanna try the same approach: https://github.com/victorb/prompta

[smrq]

I wonder if people in general would have a healthier understanding of LLMs if this mode of interaction was more common. Perhaps it would be more clear that the LLM is a very souped up autocomplete, instead of some other mind to debate.

[freehorse]

Yes I think the same. It really felt demystifying the experience for me when I first tried zed, and I already had that belief about LLMs. But when you use an LLM through a normal chat interface, it feels different, and they are also tuned to feel that way, like talking to somebody. Maybe this is why this approach is avoided by those who have some stake on AI, even if it is better UX.

[refsab]

I've found the most useful LLM UIs for me are tree-like with lots of branches where you go back and forth between your prompts. You branch off anywhere, edit top or leafs as you go.

If one branch doesn't work out you go back to the last node that gave good results or the top and create another branch with a different prompt from.

Or if you want to ask something in a different direction but don't want all the baggage from recent nodes.

Example: https://exoloom.io/trees

[sgillen]

I still think there is value in chats and retaining context. But there is also value in starting clean when necessary. Giving users control and teaching people how to use it is the way IMO.

[nomel]

The problem with retaining context is that it gets polluted. That pollution gets you into a latent space with errors, which probably not where you want your next token prediction to be sourced.

The reasonable alternative is a chat interface that lets you edit any text, the AI response or your prompts, and regenerate from any point. This is why I use the API "playground" interfaces or something like LibreChat. Deepseek at least has prompt editing/regeneration.

[diggan]

> This means the leading UI for LLMs - the chat - is the wrong UI

For coding, I'd agree. But seemingly people use LLMs for more than that, but I don't have any experience myself. But I agree with the idea that we haven't found the right UX for programming with LLMs yet. I'm getting even worse results with Aider, Cursor and all of those, than just my approach outlined above, so that doesn't seem like the right way either.