[adamklec]

I agree that's an interesting comparison to make but I'm not sure of the answer. The original purpose of this work was not to generate word vectors but rather to evaluate whether we have enough data to start using deep learning algorithms. That an RNN trained on our data was able to learn word vectors with a significant amount of structure seems like a positive sign. But I don't know how the quality of these word vectors would compare to vectors generated by more standard word2vec algorithms.

[nicklo]

There are tons of ways to evaluate word vector quality! Word analogy tasks, word similarity tasks, contextual prediction tasks, etc.

This link contains a bunch of relevant evaluation datasets and benchmarks obtained using word2vec, GloVe, etc. You can evaluate your RNN-learned vectors and compare them to a traditionally trained word2vec-trained vectors. Link here: http://www.bigdatalab.ac.cn/benchmark/bm/Domain?domain=Word%...

For more background on evaluating word vectors check out these pretty great lecture notes from Socher's NLP class: http://cs224d.stanford.edu/lecture_notes/LectureNotes2.pdf

Also, here's the original papers from a few years ago that introduced many of these datasets and evaluation standards:

https://papers.nips.cc/paper/5021-distributed-representation...

http://www.cs.cmu.edu/~mfaruqui/papers/acl14-vecdemo.pdf