[wenhan_zhou]

Yep. Or even better, compact after a random number of turns. The model must then learn to preserve useful context at arbitrary context lengths.