We've had the same thought, and are experimenting in this direction in the context of recursive language models.
Let us know if you have feedback!