Y
Hacker News
new
|
ask
|
show
|
jobs
by
Kalabasa
996 days ago
How does that fare for non-English queries? E.g. are two chars still not enough for Chinese languages?
1 comments
meepmorp
996 days ago
I've seen 1.5 chars/token used as a rule of thumb for estimating token counts in Chinese text.
link