[imh]

But that takes ages to train!

[nl]

So something like Jason Weston's state-of-the-art attention-NN based sentence summarizer took ~4 days to train.

You'd easily spend that time doing manual feature engineering just to build a baseline system.