Hacker News new | ask | show | jobs
by guhcampos 14 days ago
Isn't that what "Attention is all you need" was about anyway? Does not sound like news to me.
2 comments

It possibly goes a little further, though, since we’re talking about multi-step agentic loops here. So in addition to next token prediction there are of iterations of collecting input from outside sources.

I’m still hesitant to interpret this as “thinking ahead” without at least seeing some more back-and-forth in the literature first, though. This just seems like one of those spots where it makes sense to give other researchers some time to come up with additional hypotheses to explain the observations instead of focusing on the first one anyone proposes in isolation.

This is all obvious and not news, but there's a lot of people doing fighting retreats against LLM intelligence where the degree of obviousness matters.
Anticipatory token selection was obvious when you used structured output on gpt 3.5.

The model would construct/hallucinate pre-conditions to satisfy some final output value that the model was predisposed to.