I would say keras or mxnet for speed and production. PyTorch for research. By this point there are hardly any cases when it’s worth it to descend to lower TensorFlow levels.
Keras only offers standard layers but you can implement your own LSTM layer and use it with Keras. This way you can take advantage of all the other features
If I had to summarise the frameworks in a few words, they would be: Keras for speed, Tensorflow for production, Pytorch for research.