[mikeve]

Not sure if I fully understand it, but this seems highly inefficient?

Instead of using embeddings which are easy to make a cheap to compare, you use summarized sections of documents and process them with an LLM? LLM's are slower and more expensive to run.

[falcor84]

If this is used as an important tool call for an AI agent that preforms many other calls, then it's likely that the added cost and latency would be negligible compared to the benefit of significantly improved retrieval. As an analogy, for a small task you're often ok with just going over the first few search results, but to prepare for a large project, you might want to spend an afternoon researching.

[page_index]

In specific domains, accuracy matters more than than speed. Document structure and reasoning bring better retrieval than semantic search which retrieves "similar" but not "relevant" results.

[CuriouslyC]

The idea this person is trying for is a LLM that explores the codebase using the source graph in the way a human might, by control+clicking in idea/vscode to go to definition, searching for usages of a function, etc. It actually does work, other systems use it as well, though they have the main agent performing the codebase walk rather than delegate to a "codebase walker" agent.

[SV_BubbleTime]

My concern would be that a function called setup() might mask some really important thing, likewise a “preface” chapter might get missed by an LLM when you ask some specially deep question.

Either way that your input data structure could build bad summaries that the LLM misses with.

[mingtianzhang]

I think it only needs to generate the tree once before retrieval, and it doesn’t require any external model at query time. The indexing may take some time upfront, but retrieval is then very fast and cost-free.