Hacker News new | ask | show | jobs
by lrei 3479 days ago
> I'm surprised (and concerned) that there haven't been any retractions or errata published for the paper cited

there is a subsequent paper by Mikolov and Mesnil at ICLR with the correct results. And code to replicate them.

Retraction? You want to retract a great paper because there is a 1-3% accuracy discrepancy for a result on a more or less random text classification task?

1 comments

No, I think errata would be more appropriate; the gist of my comment was in support of the idea behind the paper, but it's disingenuous to pretend that the model works as well as described on that task.

It's also disingenuous to trivialize the difference as 1-3% - that difference is more than a third of the actual error rate (7.42% vs. 11.27%) assuming you're referring to [1]. True, the IMDB dataset isn't a very important one, but it's important to clarify when you've made a mistake, especially if your paper has been cited hundreds of times.

[1] https://arxiv.org/pdf/1412.5335v7.pdf