[atq2119]

Doesn't that argument make the fundamentally incorrect assumption that the space of produced output sequence has pockets where all output sequence with a certain prefix are incorrect?

Design your output space in such way that every prefix has a correct completion and this simplistic argument no longer applies. Humans do this in practice by saying "hold on, I was wrong, here's what's right".

Of course, there's still a question of whether you can get the probability mass of correct outputs large enough.

[marcosdumay]

How do you do this in something where the only memory is the last few things it said or heard?