[kingkongjaffa]

We have a RAG powered product in production right now used by thousands of users.

RAG is part of the solution, it provides the required style, formatting and subject matter idiosyncrasies of the domain.

But it isn't enough to do (prompt + RAG query on that prompt) alone, we have a handwritten series of prompts, so the user input is just one step in a branching decision tree of deciding which prompts to apply, in sequence (prompt 1 output = prompt 2 input) and also composition (deciding to combine prompt (3 + 5, but not prompt 4)) for example.

[TXTOS]

Totally agree, RAG by itself isn’t enough — especially when users don’t follow the script.

We’ve seen similar pain: one-shot retrieval works great in perfect lab settings, then collapses once you let in real humans asking weird followups like

“do that again but with grandma’s style” and suddenly your context window looks like a Salvador Dali painting.

That branching tree approach you mentioned — composing prompt→prompt→query in a structured cascade — is underrated genius. We ended up building something similar, but layered a semantic engine on top to decide which prompt chain deserves to exist in that moment, not just statically prewiring them.

It’s duct tape + divination right now. But hey — the thing kinda works.

Appreciate your battle-tested insight — makes me feel slightly less insane.