[jondwillis]

1) What do you mean by LLM output inconsistencies? Most LLM responses are parsed, and then if that fails, an attempt to auto-fix them is made by re-running the previous output through a rewriting/schema prompt.

2) I want that feature too, and have it planned! I want to have a sort of knowledge / progress dashboard, where users can "chat their data". I also want to add to each sub-task the ability to restart from that point. Essentially, since the project is a running on an entirely serverless architecture, this means serializing everything important, canceling current functions, and then re-hydrating from a certain point and calling the serverless functions again.

[pchunduri6]

1) While building this system, I found that the LLM can sometimes generate unpredictable responses. For example, the LLM sometimes chooses to summarize the document even for a simple retrieval question. When using expensive LLM models, this mistake could result in 10x higher cost. In your case, the LLM could generate sub-tasks that incur significant operating overheads. Just curious if you're currently facing such issues and if you have plans to mitigate them.

2) The restart idea is neat! I often faced this scenario where only few sub-questions have some issues that need to be fixed. Tweaking them without re-running the whole pipeline seems like a useful feature in this case.