[shri_krishna]

100k context limit is still a limit (we have no idea how Anthropic is achieving this - if it is extension of the base model context limit itself or some vector db trickery in the backend or probably even RAG). Even in this example, though it could fit entire text of Great Gatsby it still is 1 book/text/document. Typical business use cases require searching through hundreds if not thousands of documents/books and finding similar vector embeddings through all of them and fetching top-K results (this is how Google search works when it has to scan through embeddings for billions of websites). These top-K results can be stuffed into the 100k context limit and produce an even more holistic picture rather than just stuff one book/pdf/file into the context. Depends on the requirements though. I don't see how it might affect vector db vendors who can process billions of vectors per query and provide top-K results.

Also having a massive context length is not necessarily a good thing from perspective of cost. It also doesn't work great with a chatbot as you will have to feed the same 100k worth context back into the chatbot for every question which will turn out to be very expensive. At some point you will have to discard some parts of the context to be specific to the question being asked and that is where vector embeddings come into play. For one off research/Q&A 100k limit works great!