[caeril]

They don't now. No FF-LLMs do, simply because of their architecture.

But eventually they (RNNs, likely) will. And we won't know when.