In t-SNE, the distances in the feature vector space are preserved in the projected space. IIRC, these distances serve as boundary conditions to a stochastic diffusion problem. The actual positions and the orientation are allowed to be free variables.