[PartiallyTyped]

I'd argue that humans are by definition autoregressive "models", and we can change our minds mid thought as we process logical arguments. The issue around small errors accumulating makes sense if there is no sense of evaluation and recovery, but clearly, both evaluation and recovery is done.

Of course, this usually requires the human to have some sense of humility and admit their mistakes.

I wonder, what if we trained more models with data that self-heals or recovers mid sentence?