[eginhard]

From LSA/SVD you get a V x K matrix as well - that's exactly what the factorisation is doing.

The following two papers also go into detail about the mathematical similarities between LSA and neural embeddings and achieving similar performance with both:

Levy, O. and Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. https://www.cs.bgu.ac.il/~yoavg/publications/nips2014pmi.pdf

Levy, O., Goldberg, Y., and Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. http://www.anthology.aclweb.org/Q/Q15/Q15-1016.pdf

[twelfthnight]

True, you could use the right matrix of the SVD on the term document matrix. As far as I know, the embeddings won't have the same interpretation as the left matrix from SVD on the word-context matrix. (By LSA I mean SVD on the term document matrix).

Those are excellent papers, by the way.