[jderick]

Can it generate sequences of emoji that is has not seen before?

[wxs]

Yes it can, language models can generate phrases that have not been seen before. Karpathy's RNN post is a pretty good intro to how that works http://karpathy.github.io/2015/05/21/rnn-effectiveness/