[ftxbro]

So they have had LLMs with small contexts like one or two words or a dozen letters for a long time, ever since like Laplace or Shannon or Markov. They were called Markov chains. No one really guessed this (although it was known to be theoretically possible in the sense of ai-completeness), but it turns out that longer ones turn out to even in practice unlock so many cognitive capabilities bordering on superhuman. If this is the main difference between the Markov chains that they have been using for autocomplete for decades versus the ones that will beat you at the GREs or the bar exams or every AP test, then it is natural they are curious what happens when they make the context even longer.

[stevemadere]

No specific practical problems though? Looks to me a lot like "it's amazing. we want more amazing" rather than "if we had it, we could solve this specific practical problem people have been wanting to solve for a long time without considering a LLM as a possible solution"