[halflings]

Ah, yes, Google are not taken seriously in academia because they are not releasing the code source to their models. Google is usually the largest contributor to most ML conferences (NIPS, ICML, etc.)

Most papers are not about implementation and more about the concepts or proofs. They are rather straightforward to reimplement, and I don't think anybody is accusing them of faking their results.

[throwaway466453]

> Google is usually the largest contributor to most ML conferences (NIPS, ICML, etc.)

That is because of their overwhelming influence, not the quality of their publications.

> They are rather straightforward to reimplement

Le and Mikolov's "Distributed Representations of Sentences and Documents", frequently cited as the original example of "doc2vec", could not be reproduced by Mikolov himself. [1]

> and I don't think anybody is accusing them of faking their results.

They sure aren't. That, too, is because of their overwhelming influence. You have to say very nicely that their results are wrong.

For example, here's an IBM research paper that leads and concludes with "we reimplemented doc2vec and made it work well", and whispers "but not as well as Le said". [2]

[1] https://stats.stackexchange.com/questions/123562/has-the-rep...

[2] https://arxiv.org/pdf/1607.05368.pdf

[nl]

I've been working on doc2vec stuff recently.

The statement Le and Mikolov's "Distributed Representations of Sentences and Documents", frequently cited as the original example of "doc2vec", could not be reproduced by Mikolov himself. is an overstatement - there was only one part that couldn't be completely reproduced.

It's true that Quoc Le's results on the dmpv version of doc2vec have been hard to reproduce. However, the very stackexchange link you cite above points out that it can be reproduced by not shuffling the data. It's likely that this was an oversight.

However - and it's an important thing - the reason this example gets some attention is because doc2vec is a very strong model even in dbow form.

here's an IBM research paper that leads and concludes with "we reimplemented doc2vec and made it work well"

No, they took the Gensim doc2vec implementation and experimented with parameters on different datasets[1].

Also, Mikolov's Word2Vec work was even more important than doc2vec and was fully reproducible and was released with code and trained models, while at Google.

[1] https://github.com/jhlau/doc2vec

[abhaga]

> They are rather straightforward to reimplement

Not really. Very often you will find the crucial details that make it work missing. Not sure if things have vastly improved over past 5-6 years.