I also have an article on RNNs, although it's focused on explaining a special version, called an LSTM: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
If you have experience with functional programming, you might find this a nice way to think about Conv Nets/RNNs/etc: http://colah.github.io/posts/2015-09-NN-Types-FP/