[machiaweliczny]

Google scaled context to 40K tokens

[contravariant]

Using tokens as context still sounds to me like you're asking someone to read back text that someone else wrote and continue the story. It might work but it's not the best way to get a coherent narrative.

[inciampati]

How can you have a coherent narrative if you can't link things across very large contexts?

[contravariant]

I'm saying the context should consist of more than just tokens.