Hacker News new | ask | show | jobs
by janwilmake 357 days ago
You know what's also faster to roughly get the amount of tokens? string.length/5
1 comments
_flux 357 days ago
It is not helpful in actual tokenization, though.
link