[Bahushruth]

That’s really interesting, especially your point about preprocessing and indexing being required to make search outputs good. What was the first sign that made you realize querying live APIs wasn’t enough?

Was it latency, missing data, or just that results weren’t relevant? And when you say preprocessing, what kind of transformations or normalization ended up being most important?

[rogerkirkness]

Keywords or vector search on their own don't get good results for high entropy queries. MCP type approach is good for low entropy things like fact-based single source answers. [1]

[1]: https://arxiv.org/abs/2504.07106