[nl]

Just learn Deep Learning instead.

I'm a NLP person, and I think the Wit.ai people said it best:

Many papers were kind of “the state of the art for X was Y. We replaced the hand-crafted, manually hacked, heavily engineered Z by a RNN. It improved state of the art by 5 points.” The poor guys who presented deep learning-free papers invariably got the question: “did you also try with a [insert deep net technique here]?”[1]

The only downside with this is that traditional NLP tools are still probably easier to use, and you'll usually need to understand vocabulary to be able to talk to other people about your problems.

[1] https://wit.ai/blog/2015/09/23/emnlp

[imh]

Wouldn't this require larger datasets? That isn't always an option. I'm imagining that a smaller, more computationally efficient network could learn nearly as well with fewer data points given these heavily engineered features. Is that off base?

[nl]

Basically, no. See http://karpathy.github.io/2015/05/21/rnn-effectiveness/

He gets pretty amazing results with a corpus size around 10M.

[imh]

But that takes ages to train!

[nl]

So something like Jason Weston's state-of-the-art attention-NN based sentence summarizer took ~4 days to train.

You'd easily spend that time doing manual feature engineering just to build a baseline system.