I believe that illusion is based on our visual processor's reliance on local contrast in order to adjust the perceived true color to the inferred lighting conditions. So local adjacency information here is what is being used, rather than just temporal recently.
Similar to those "these boxes/squares are the same color" illusions.