Would love to see a system that blends cheap lexical (Fulltext Search) or semantic/vector search using SQLite and chooses the best approach given the input.
If you want the best possible solution vertical for most business, I'd be looking at using Lucene for FTS duty.
Having the FTS engine provide a google-style snippet of the most relevant document chunk is the holy grail for RAG applications. Lucene does this kind of thing better than anyone else:
Having the FTS engine provide a google-style snippet of the most relevant document chunk is the holy grail for RAG applications. Lucene does this kind of thing better than anyone else:
https://lucene.apache.org/core/8_0_0/highlighter/org/apache/...
It is also very easy to customize this engine and align the document tokenization & indexing concerns with your specific retrieval scenarios.