[pizza]

Seems like we should just use gradual annealing of tokens to more fine grained single character tokens over the course of training then

[kevmo314]

I believe that's similar to the idea behind https://github.com/facebookresearch/blt