Hacker News new | ask | show | jobs
by gradys 1164 days ago
The model is indeed so overpowered that it doesn’t matter in practice. See the Sentencepiece paper for some discussion of the design decisions on stuff like whitespace.