[jackpirate]

This is false. Standard sampling algorithms like beamsearch can "backtrack" and are widely used in generative language models.

It is true that the runtime of these algorithms is exponential in the length of the sequence, and so lots of heuristics are used to reduce this runtime in practice, and this limits the "backtracking" ability. But this limitation is purely for computational convenience's sake and not something inherent in the model.