[mdp2021]

> multi token prediction of sufficient length

Is multi token prediction the same as predicting the embedding of a complex token (the articulation of those input tokens in a sentence)?

[blackeyeblitzar]

To be honest I don’t know. Maybe the only way to know is to build and measure all these variations.