I am wondering how the search latentcy will be with your approach, especially for cases with more than a few hundreds documents. Do you have any insights about that?
Actually a few hundred documents is really no biggy, my current benchmarks is in the range of <250ms (instant feeling) for hundreds of thousands of paragraphs.
I'm testing this on a large knowledge base.