[OtherShrezzing]

How do you deal with context length degradation here?

The harder questions will only arrive when the context is getting full.

[danieljacksonno]

If you have it question you for 1M tokens (aka the full length of the Wheel of Time series), I think your own context might get full before the LLMs.

[mkl]

The Wheel of Time is 4.3M words: https://www.reddit.com/r/WoT/comments/zs1ust/chart_of_wot_wo.... Some of its individual books are near 400k.

[d4rkp4ttern]

Right, even a conservative 200k context length is on the order of 200 pages, which is more than enough context to arrive at an answer.

[raincole]

If it's a well-known concept (like pretty much anything you can find from undergraduate textbook), the LLM doesn't need the whole context to teach you.

If it's something actually novel, no matter how much context you provide it'll still hallucinate.