Hacker News new | ask | show | jobs
by Vecr 537 days ago
It's not optimal, but if you want to play it safe you can use 1 byte = 1 token, plus a few control tokens like stop, user turn, model turn, etc.