Hacker News new | ask | show | jobs
by imh 3924 days ago
I think NLP is really cool, but it seems to be moving so quickly. If I wanted to get a decent overview, are there some review papers or textbooks with good coverage that aren't too out of date?
1 comments

Just learn Deep Learning instead.

I'm a NLP person, and I think the Wit.ai people said it best:

Many papers were kind of “the state of the art for X was Y. We replaced the hand-crafted, manually hacked, heavily engineered Z by a RNN. It improved state of the art by 5 points.” The poor guys who presented deep learning-free papers invariably got the question: “did you also try with a [insert deep net technique here]?”[1]

The only downside with this is that traditional NLP tools are still probably easier to use, and you'll usually need to understand vocabulary to be able to talk to other people about your problems.

[1] https://wit.ai/blog/2015/09/23/emnlp

Wouldn't this require larger datasets? That isn't always an option. I'm imagining that a smaller, more computationally efficient network could learn nearly as well with fewer data points given these heavily engineered features. Is that off base?
Basically, no. See http://karpathy.github.io/2015/05/21/rnn-effectiveness/

He gets pretty amazing results with a corpus size around 10M.

But that takes ages to train!
So something like Jason Weston's state-of-the-art attention-NN based sentence summarizer took ~4 days to train.

You'd easily spend that time doing manual feature engineering just to build a baseline system.