[chaxor]

There isn't a DFT directly, it's a more obvious statement here. The circulant matrix (linear graph of words) always has the same eigenvectors and is diagonalized via DFT.

The PE in original Viswani is based on this, they just didn't put in all the details. So effectively the model gets hints from the PE that it's a linear graph because these are the eigenvectors.