[snowfield]

Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process.

But if all you wanted is a search engine that's a bit easier.

The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?

[boredemployee]

I think it depends of what they want. Like a search is indeed an easy solution, but if they want a summarization or a generated, straight answer so then things get a little bit harder.

[cyanydeez]

A solution that combines RAG and function calling could span the correct depth, but yeah, the context depth is what will determine usefulness for user interaction.

[HeavyStorm]

The LLM would have to be trained on the local data. Not impossible, but maybe too costly?

[snowfield]

It sounds nice in theory but your dataset is most likely too small for the LLM to "learn" anything.

[IanCal]

I'd like to play with giving it more turns. When answering a question the note interesting ones require searching, reading, then searching again, reading more etc.

[rgrieselhuber]

This gets to the heart of it. Humans are good at keeping a working memory, as a group or individuals, as lore.