[tk90]

Isn't there retrieval degradation with such a large context size? I would still think that a RAG system on 128K is still better than No Rag + 1M context window, no? (assuming text only)

[cathalc]

Absolutely. Gemini results tend to drop off after 128k tokens according to RULER: https://github.com/hsiehjackson/RULER

[ayyndrew]

If I'm reading the paper right, it says Gemini 1.5 is good up to 128k, and RULER doesn't test beyond 128k

[cathalc]

Hmm perhaps you are right.. There's no context on the page about what ">128k" means so possibly a bad assumption on my part!

[afro88]

Not sure why you've been downvoted. Needle in a haystack testing exists for a reason