[valine]

So instead of next token prediction its next event prediction. At some point this just loops around and we're back to teaching models to predict the next token in the sequence.

[lumost]

Tokens are an awfully convenient way to describe an event.

[phyalow]

Tokens are just discretized state representations.

[ww520]

It’s the next state. So instead of spitting out words, it will spit out a whole movie, or a sequence of world states in a game or simulation.