[falcor84]

If this is used as an important tool call for an AI agent that preforms many other calls, then it's likely that the added cost and latency would be negligible compared to the benefit of significantly improved retrieval. As an analogy, for a small task you're often ok with just going over the first few search results, but to prepare for a large project, you might want to spend an afternoon researching.

[page_index]

In specific domains, accuracy matters more than than speed. Document structure and reasoning bring better retrieval than semantic search which retrieves "similar" but not "relevant" results.