[ricardobeat]

Love this kind of experiment. Would the model perform better with word tokens?

[encrux]

A friend of mine forked the repo and tried it with BPE (Byte-Pair Encodings), and it did noticeably improve performance.