Hacker News new | ask | show | jobs
by kardos 1 day ago
French text is somewhere around 10-30% longer than the corresponding English text. I would guess much of what you save on smaller vocabulary is lost on the lengthened text.
1 comments

What would matter more is the token length, depends how well your tokenizer was trained on french I guess.