Hacker News new | ask | show | jobs
by weird-eye-issue 991 days ago
It's done by tokens. You have to convert the text to tokens... Some languages are 20x different from these estimates