One argument for SVD is the low reliability (as in results fluctuate with repeated experiments) of word2vec embeddings, which hampers (qualitative) interpretation of the resulting embedding spaces, see: http://www.aclweb.org/anthology/C/C16/C16-1262.pdf