[zwaps]

Gemini loses coherence and reasoning ability well before the chat hits the context limitations, and according to this report, it is the best model on several dimensions.

Long story short: Context engineering is still king, RAG is not dead

[deadbabe]

RAG was never going away, the people who say that are the same types who say software engineers will be totally replaced with AI.

LLMs will need RAG one way or another, you can hide it from the user, but it still must be there.

[tvshtr]

Yep, it can decohere really badly with bigger context. It's not only context related though. Sometimes it can lose focus early on in a way that is impossible to get it back on track.

[risyachka]

Yep. The easiest way to tell someone has no experience with LLMs is if they say “RAG is dead”

[apwell23]

> someone has no experience with LLMs

Thats 99% of coders. No need to gatekeep.

[Xmd5a]

Gemini loses the notion of context the longer its context is: I often ask it to provide a summary of our discussion for the outside world and it will reference ideas or documents without introducing them, via anaphore, as if the outside world had knowledge of the context.

[Inviz]

Cursor lifted "Start a new chat" limitation on gemini and i'm actually now enjoying keeping longer sessions within one window, becuase it's still very reasonable at recall, but doesnt need to restate everything each time

[darepublic]

Can you elaborate on how prompts enhanced with rag avoid this context pollution? I don't understand why that would be