[nicklo]

There are tons of ways to evaluate word vector quality! Word analogy tasks, word similarity tasks, contextual prediction tasks, etc.

This link contains a bunch of relevant evaluation datasets and benchmarks obtained using word2vec, GloVe, etc. You can evaluate your RNN-learned vectors and compare them to a traditionally trained word2vec-trained vectors. Link here: http://www.bigdatalab.ac.cn/benchmark/bm/Domain?domain=Word%...

For more background on evaluating word vectors check out these pretty great lecture notes from Socher's NLP class: http://cs224d.stanford.edu/lecture_notes/LectureNotes2.pdf

Also, here's the original papers from a few years ago that introduced many of these datasets and evaluation standards:

https://papers.nips.cc/paper/5021-distributed-representation...

http://www.cs.cmu.edu/~mfaruqui/papers/acl14-vecdemo.pdf