Hacker News new | ask | show | jobs
by xaphod 1260 days ago
Text is broken into something called "tokens" [0] for GPT. It doesn't work on individual characters. That explains why it always fails these character counting questions.

[0]: https://help.openai.com/en/articles/4936856-what-are-tokens-...

2 comments

Interestingly both “halves” of the word as it split it have the same number of tokens
>it always fails

It usually is able to get the answer within 1 character of the actual answer.