Hacker News new | ask | show | jobs
by janwilmake 357 days ago
You know what's also faster to roughly get the amount of tokens? string.length/5
1 comments

It is not helpful in actual tokenization, though.