[blackeyeblitzar]

Isn’t “sentence prediction” roughly the same as multi token prediction of sufficient length? In the end are we just talking about a change to hyper parameters or maybe a new hyper parameter that controls the granularity of “prediction length”?

[mdp2021]

> multi token prediction of sufficient length

Is multi token prediction the same as predicting the embedding of a complex token (the articulation of those input tokens in a sentence)?

[blackeyeblitzar]

To be honest I don’t know. Maybe the only way to know is to build and measure all these variations.