Hacker News new | ask | show | jobs
by hungrigekatze 1096 days ago
For some discussion on how to have the LLaMa tokenizer (properly) handle repeating spaces, please see this discussion: https://github.com/openlm-research/open_llama/issues/40