[jampekka]

> Modern BERT with the extended context has solved natural language web search.

I doubt this. Embedding models are no panacea even with a lot simpler retrieval tasks like RAG.

[noosphr]

RAG is literally what Google Search is.

Unlike the natural language queries that RAG has to deal with, Google searches are (usually) atomic ideas and encoder-only models have a much easier time with them.